A lot of folks rightfully warn about too much world building/lore (especially if crammed into the story itself- it's totally okay to think about and develop these things without it all being explicitly being present!) but I'll offer the experience of someone on the other side of the stick, not enough world building.
A pitfall that I'm experiencing with my current comic is that the world building is rather half-baked in a lot of ways and thus I don't think I'll be continuing the series after I complete the upcoming season/arc. The reason being: I initially wrote this world simply as a vessel to tell a short one-shot and get my feet wet in webcomics, so like literally everything about the world was designed to service this specific story and I also conceptualized the whole thing in a pretty short span of time, like 2-3 weeks maybe?
But what I'm finding now as I write the next part is that there's just a lot of inconsistencies or luke-warm ideas that I imagine will only continue to stand out more the further I go. I can't help but feel as though if I had spent some more time just setting up the basic rules and concepts of this world the whole thing might feel more cohesive... but alas it was always kinda meant to be a "one and done" kind of deal lol.
So yeah, I think that's what I'll take moving forward before starting another fantasy project- even if it's for a short story- is to just spend a little more time setting up a solid foundation to build upon. I think it's still fine to explore and develop intricacies as you go- I could never be one of those authors that literally develops the entire world before getting started, I think- but you have to have solid ideas and rules in place first to make everything feel cohesive 