the only reason to be even remotely worried about this is if your series is specifically aimed at kids or you have a particularly interest in making 100% sure it's family-friendly and all-ages. Even then, that's not an instant death sentence for the idea of doing NSFW work, it's just worth thinking about and a good idea to keep it more explicitly cordoned off. Keeping it confined to Patreon is a very common tactic both for this and to gain more traction, sex sells and all that.
Overall though, no, absolutely not, do not give a shit whatsoever. Even if you're drawing 'objectionable' fetishy stuff, nobody with half a brain is going to try to cancel you over its mere existence.
The only thing I can think of is if you're drawing NSFW of underage characters. I may think it's gross, but it's also fictional, so I personally wouldn't try to cancel or deplatform an artist who draws that stuff. Unfortunately, my stance on that is not as broadly shared and it may present a problem for you if you did it, which you've specifically stated you aren't doing, so I would not concern myself with this in the slightest.