Not at all. I think you might just be consuming a lot of media that doesn't really want to take itself seriously. Stories for kids, or stories that are super campy (i.e. more interested in portraying a 'fun' villain than an efficient one) or stories that have a runtime to fill and don't mind dragging their feet a bit. ^^; None of these things are inherently bad or anything, but if you just want the villains to start upping the body count and get sh!t done, then you're gonna walk away unsatisfied.
I think probably the most maddening instance of this in my memory was in RWBY, when the heroes accidentally run into the 'final boss' of the series, an immortal witch who's killed countless innocents and supposedly cares for nothing and no one...only for all of them to walk away unharmed and unscathed, ¯_(ツ)_/¯
And everyone else was gushing over this brief encounter/pseudo-battle, like "oh my gosh, none of their attacks can hurt her! She could have killed them all easily; isn't she so scary!?!???"
And meanwhile I'm sitting here like "no, because she didn't even try." T_T
I didn't expect the main characters to get killed, but I did expect some amount of harm to come to them. Broken bones, bruises, bleeding wounds. At the very least, fear, screaming, trauma. But nope, the villain was conveniently distracted enough to just let them all go. ¯_(ツ)/¯ Even though she doesn't actually know them and would have no reason not to kill them on sight. ¯_(ツ)/¯ Just have her hold them down and vamp for a bit, and that'll be our uber-scary sneak peek at our final boss. ¯_(ツ)/¯ ¯_(ツ)/¯ Don't try to raise the stakes of the show or anything...
That's the scene that convinced me that RWBY was essentially just a kids' show pretending to be 'mature'. And there's a lot of media like that out there, covering the screen in blood and leaning on the artists and actors to create a scary atmosphere...while at the same time, clinging desperately to the status quo and preventing actions from having consequences, because they want to keep the plot easy to write and digest, and keep their main cast intact and unchanged for marketing purposes. Fortunately, these things become easier to spot the more you run into 'em. :T
I don't really know what sort of media you lean towards, but I have some recommendations for stories with villains that actually try to win, if you're interested. I'll list 'em in descending order of maturity:
Anime: Noir, KARAS, Rozen Maiden, Senki Zesshou Symphogear*
Live-Action Drama: Nikita, The Mandalorian (S1-2)
American Toon: Avatar: The Last Airbender, Avatar: Legend of Korra, Infinity Train, Transformers Prime, Ben-10 (original series)
...Yeah, these lists are short, but that's only because not everything always has a clear "villain" who wants to hurt people (hence why the longest list is 'American Toon'...my recent favorite Live-Action Drama and Anime aren't structured that way, so despite being good they're not on the lists ^^; ). I hope this helps, and that you find some villains you like someday.
*I'm always iffy about recommending Symphogear due to all the fanservice, but I can't deny that it's an action-packed show that does not waste time. I was looking back through the first 2 seasons the other day, and I was amazed at how much the plot progresses in every episode...they really don't make magical girl anime like that anymore...