Cats haven't been fully domesticated by humans, unlike dogs who we evolved with over the thousands of years. You can argue that they are, but they can turn feral (i.e wild) much quicker then other animals cause they haven't had that reliance and trust with humans bred into them. They are still wild at heart.
Having any pet let outside to roam is bad for many reasons, but any non-native animal released into an environment is devastating to the entire ecosystem.
Yes, cats are wild and hunt and that is all normal. But, they are not native hunters and if not helping to wipe out already endangered birds or other wildlife, they are taking the food out of the mouths of native predators. Yeah, they might hunt rodents and other pests, but they don't solely do so. They hunt whatever they can find, which can be animals that are beneficial (lizards that eat snails off plants or birds that pollinate your plants).
Everything has a reaction, direct or not.
Even if there isn't nothing endangered or at risk near you, eventually it could be if everyone who has a cat lets it out. Like humans hunting an animal to near extinction, it doesn't happen in our life time. But every animal the cat kills, is one less pair to breed. Times that by all the outdoor cats and then add a hundred years or so, it can mostly likely be a problem, even something that might seem numerous to us now.
Keep your pets inside. If not, let them out in a control environment (In your backyard with cat fencing installed, on a leash in a safe environment, or in a cat enclosure). Treat you cat being let outside as if you would a dog. Leash it. Control it. Have responsibility over it.
In this day and age, there is so many things at your disposal there isn't any excuses to continue to let you cat outside. They can get all the exercise they need from you playing with them and allowed to free roam in a controlled environment. Dogs do it all the time, and have much higher energy reserves then a cat does.
And yes, some cats are really chilled and that all good. But you still wouldn't accept a big German Shepard being unleashed in a park of children, even if his owner said he is good with children. All animals can be unpredictable.