Isn't the basic idea of a faction segregation, even if they aren't strict about the rules, the existence of a faction creates a mental barrier at the very least, at least in my experience. (Not saying that it has to be the case in your story, just thinking about in life.)
I find this especially true with the idea of marrying outside your culture. While it may not be illegal any more (at least not in most countries) the fact that if you do this people will comment on you dating or marrying 'outside your race' creates a mental divide between one group and the other, as different 'kinds' of people, rather than just "people."
Just a thought. I at least know this is how comments like this have always appeared in my life. Though I haven't written a lot of stories that address that idea (and maybe I will in future.)