Do what you find more comfortable.
I originally replied to comments even if they aren't many but right now I just let them be, mostly because I'm not the author of the series so I don't think it's a great impact if I reply or not, as well, part of the reason as to why I don't reply anymore is that readers take advantage of that just to be pushy and demand more episodes, to hurry up, or that "I'm too slow so I should focus on drawing instead of replying to them".