All the darn time! I can't leave most panels alone - in the end I just have to say to myself 'right, you've tinkered too much now, let it go' and put it one side. Otherwise I'd never get anything finished!
But I think that's part of growing as an artist and so long as you can see what's making you feel a bit squiffy about it, then you can work on that the next time
Besides, you say you've been improving recently, and that's all that matters - keep it up!