We use short hand of our own society to explain why our fantasy society is the way it is. Hence why fantasy medieval worlds have castles, even though as soon as you have dragons and other flying creatures they would be obsolete. It's why those same setting use absolutely dumb timelines of thousands of years but somehow they are still using swords and shields in combat. But as long as you say "it's like japan when samurai exist" most of the ground work is done. The problem is when you start doing things "because that is how it works in my world" that people start asking why. Take for example the men in dresses. If the men in your world wear dresses (I'm talking frilly looking stuff, not just tunics) because in your world they just do, this is not going to be a good enough answer. The audience doesn't want to hear that as it's basically saying "because".
That easy out works on children's books (Harry Potter) and the like because you're not suppose to think too deeply about them. For a more mature book, you need explanations that work. The more fantastical the setting, the easier the explanation. Star Trek made a career out of dumb, non-science explanations but they did give them.
When it comes to characters, it has the same thing. Until you have explain or shown why a person will do a thing, the audience needs that personality to fit what you visually described it to be. If the drunk is suddenly super smart, tell people why. IF the smart person is suddenly a fool, tell them why. There is a social contract between reader and writer that you can only break by explaining why it is different than what you have put out there at first.