Ah I see, I know in my case, as well as my uncles case (who is much older than me) it was a slur directed at us, and is not something I would ever refer to them as due to the trauma tied to it, but is one that I am fine under as long as it's not being used to 'other' me. (I also understand that there's a lot of discourse about this word dependent on age, a lot of older members of the community don't like it, while a lot of younger ones are fine and see no issue with it)
To my understanding, it IS a valid word for queer people to refer to themselves, especially if they are comfortable with the term and it helps to identify them, but it's not a word you call someone who already associated under a different letter, even more so if they identify as something else than queer but you not refer to them such and just call them queer in a social setting. (even though it's perfectly acceptable as a general label, as an individual, unless specifically specified, it is not).
Signalling out 'the gays' and referring everyone else as 'the queers' in a sentence that refers to everyone else in the community being problematic unless they are gay IS and issue for me, and though likely not the intent it really came off as rude, like being 'othered' which again, due to my PoC background, is not something I enjoy seeing so I wanted to bring it to their attention especially if it was unintentional.
Again though, I could be wrong because honestly, it's really hard to keep up with all the changing language and tolerance in the LGBT+ or LGBTQ community because along with fighting stigmas, like the fetish sides of BL and GL, there's a lot of infighting and gate keeping which is unfortunately keeping information all over the place and very difficult for people legitimately trying to understand even when you are researching these things.