I don't think there's an issue with refinement, but outright changing whole scenarios is usually where I myself draw a line of never crossing. For me, it's more important on "How can I make what I've already written better for my readers?" rather than fully changing certain events.
As for you changing certain parts in the manga compared to the Novel, I don't think there's a problem in doing that. Many Authors do that, sometimes they use the opportunity of a Manga or Anime adaptation to do things with the Story that didn't exist in the Source Material, which can lead to a whole "2-3 Different Directions for the same Plot." Albeit, it doesn't work out well unless you've genuinely given thought of it.
Long Story Short, Be Proud of How Far You've Come, Don't fully change what you've already written, but make it better to understand or so for Newer Readers.