My story is constructed around flashbacks, and that's something I really like reading too.
In my case, I cannot avoid them as the story is structured as follow: actual time is the children's generation, and these children, little by little, learn things about their parents generations through people's memories, tales and documents (all treated as flashbacks). There are flashbacks within flashbacks, as well as several flashbacks for the same memory (remembered by different people, who perceived different things, or are more or less reliable).
The thing is, I'm always making sure that each flashback is necessary. It has to advance the plot, bring new questions; or bring a new perspective on a person or an event.
If I understand well, your flashbacks would fall in that last category: bring a different perspective on a character, and you are not sure of it's worth it.
Difficult to know. I'd say the best would be to ask someone to read both versions, (even just the script) and ask them to tell you if that changes their perception of the character (and if yes, if it changes it the way you would like).