it depends how you say it. if you're out there loudly proclaiming that you don't want to be around "females" because they're all sluts or whatever, yeah you sound like an incel. if someone asks you whether you're seeing anyone and you say "no, I honestly just prefer to be single," I can't imagine anyone would think less of you for that.