I can totally respect your concern.
From my end, I've seen white-privileged people making those remarks and openly discussing it. Joke or not, it's still waving the possibility they could very well get away with it, despite their own criticisms of accepting refugees.
As a second-class US citizen, I don't have that privilege. (I wasn't even allowed to vote for president) I could try to move there, but even my own visit to Canada left me with a bad taste in my mouth.
But if minorities have claimed the jokes, then yeah, step back and let them take it. Give oppressed groups all the power to vent and release steam.
And, on a side note, for the other people saying it was the Hilllary supporters who took to the streets, stop. Stop. It wasn't a protest to make Hillary president, it wasn't about "socialism", it was to be against the plans this horrible man (and mainly his horrible vice president) want to implement. It's about the safety of the people, not about being sore losers.