It's a tricky balance, honestly, managing the cognitive load of a reader.
With my comic, Errant, I did end up choosing to simplify the worldbuilding somewhat, or to at least make a lot of the stuff about the worldbuilding go unspoken. There are certain things I could tell you about the worldbuilding that are true... but also burdening more casual readers with them, who might already struggle to follow a story about 4-5 main characters, one of whom has two identities and another has lived multiple lifetimes under different names, across a 10 year timeskip, trying to covertly arrange a coup... might just make it hard for them to follow or engage with.
As an example, the characters in Errant do not speak English. It makes sense, right? King Arthur was a 100% real, actually magical Romano-Celtic King of "Britannia" in this universe, with a real magical sword. He fought off the Angles and Saxons and so.... there was never an "England" and English doesn't exist. So the language they're speaking isn't the Germanic-Romantic hybrid language of English... but probably a Celtic-Romantic language that'd sound like a cross between Welsh and French.
...But that's really hard for people to get their heads around, so they all speak British English in the text. When a character speaks British English, it gets across "they are British" in a way that makes it a lot easier to explore things like the personalities and class dynamics. I have at least done one little thing, which is never having the characters use any term derived too directly from Christianity, because they're a pagan society, and if they worship gods, they're probably polytheistic (Except Subo, who was probably raised Muslim). They never say "god", "jesus", "jeez", "bloody hell!", "christ", "crikey" etc. and Sarin often says "Sulis Minerva!" refering to the composite Romano-Celtic deity who combined the Celtic healing and protection goddes of water, Sulis, with the Greek/Roman goddess of war and wisdom Athena/Minerva, Honestly, I really don't think the audience cares or notices, but it makes me happy.
Realistically, the butterfly effect of the alternate dark age history, and having magic in the world ought to have made the world they live in basically unrecognisable. No England... does that mean no British Empire? What does mean for America? Is there a United States of America in my setting? Various inventions could actually be radically different because of this change, or have come out at different times... Are there mobile phones? Do they look anything like our mobile phones? Was there a WW2? And if not, how come references suggest that Sailor Moon exists, when no WW2 could mean no Tezuka making cheap comics after the war, which means no manga, which...
But I've chosen to not explore this, because every time a normal thing is unrecognisably different, it's necessary to devote time to showing context clues about what it is, or explaining what it is.
I call Excalibur Excalibur in my comic. I could have called it Caledfwylch, its original Celtic name, and did consider it... but then I'd have to have had a panel or scene where somebody or a narrator clunkily explains that Caledfwylch is Excalibur. What has that panel added? Not a lot really, now you just know that there's this sword that's basically Excalibur, but it's called Caledfwylch... so it's basically Excalibur, why does it need to be called something else, when it's just a plot item where the important thing is that it functions like Excalibur? By all logic, Excalibur should be a little roman spatha or gladius type sword, not this big fancy fantasy sword... But that's another thing to explain, because readers who don't know Roman history probably aren't aware of that and would just be like "hurr hurr, why legendary Excalibur so small and boring?".
It comes down to this for me, and some people may prioritise differently, but I personally believe that the audience has limited cognitive load, and that every new thing you need to get them to understand adds to the mental "weight" they are carrying. If I want to tell a story, and I think the most important thing is that it's about this one girl who really wanted to be a shounen manga hero, and then she grows up and it's all horrible because on another level, it's about "girlboss" culture and how modern society treats women, and it's in a setting that should feel "British", then I should make sure that those are the things somebody takes away from it, even if it means I need to cut corners on the worldbuilding to make it more familiar and not need thinking about. Cars look like cars, phones look like phones, there's a social media app called "Sqroller" that's pretty much twitter (wait...even twitter isn't twitter any more irl...ugh...) France is basically France, but it's just called "Gaul" instead, potato crisps exist, there is a Japanese culture that produces Anime, and one of them is Sailor Moon. I just don't have the space to explain a whole alternate history, or to have somebody say "let me check my phone" every time they get some weird looking device out so the audience knows that's a phone (and I have done that in another comic I did in the past where "phones" were little helicopter-like drones that project holograms, but it was a much simpler comic!).
It would be cool to make a comic some time where the alternate world IS the story. I admire Paul Duffield's The Firelight Isle for that. In the case of Errant though, the story isn't about the world so much as about these friends, and they already have a lot going on.
It's been a learning process; I'm not sure I'd necessarily do it this way if I made another Fantasy comic, but... that's how I ended up doing it.