Think about it like this - you’re in a relationship and you find someone else attractive. And maybe you feel a certain way about them. But you love your significant other, so you don’t act upon it. Or, similarly, maybe you find someone else who is attracted to your sex/gender, but just not attracted to you specifically (which is okay and says nothing about you!), so you don’t pursue it. No reason to feel guilty: you can’t control attraction!