I second that. The very premise of my story is based on the fact that magic exists and lets you do things you normally wouldn't be able to do, if you take that out of the plot, the entire story would make no sense... so no, it's definitely not just an "aesthetic" thing. I also wouldn't lump sci-fi and fantasy in the same category, as they're two very different things. They can overlap depending on the story, but that isn't always true and putting the two things together would make it hard for people to find what they're truly looking for.
Overall, I don't think I like this whole distinction between "prime" and "secondary" genres... especially since a lot of the genres I see listed as "secondary" are actual genres of their own, often with a long and well established literary tradition (poetry, anyone?), not some sub-genres or whatever, so why calling them "secondary" or using them to "refine" a research? If I'm looking for poetry in a book store I look under the poetry section, not under "drama", "action" or whatever. I find it far too convoluted and counter-intuitive for both readers and creators.
Personally, I like the current system in which we pick a main genre and then add a couple more labels to define other themes that can be found in our stories. What I'd do is adding more genres and give sub-genres to each category. So, for example, the "Fantasy" category would be left exactly where it is, but additionally you could pick a sub-genre, which can be High Fantasy, Urban Fantasy, Dark Fantasy etc. That would help sort things better, help readers find the exact sub-genre they're looking for and also make it easier for creators of specific sub-genres to be found, especially when it comes to extremely broad categories where it's easy for your work to get drowned under all the much more popular stuff. Basically, more options, rather than less.