I take a kind of 'cross that bridge when I come to it' approach to world building. If I suddenly have an urgent need to explain why something is the way it is or how something works, I go away and think about it. On the other hand, some concepts present themselves or I've been hanging onto them for a long time as a core pillar of the story and these form general ground rules from which I improvise.
With designs I feel it out characters/environments on the page. Sometimes I have a very clear idea and other times I have to play around. Jed was a doodle whose core 'look' settled over the few pages. I can't remember what early versions of Morris looked like but I know they were quite different.
In terms of world history and culture I omit anything that isn't absolutely necessary. A lot of the time readers don't need help filling in the blanks and seeing (as I do) the scope of possibilities that are presented to them. Dropping hints and details is enough, and I've never had anyone ask for more OR get the wrong idea about what they're seeing (as far as I know).
Honestly, if I thought about my world too much I'd probably end up doubting myself. There are a lot of tropes and cookie-cutter concepts that get used in Fantasy stories these days and, frankly, I can't keep track of them all. What good will it do me to sit and worry if my ideas are original enough? I mean no harm or offense to anyone. I'm just going what feels right.