For myself genre doesn't matter, its how important the information is or what you are trying to highlight. I was starting to nod along to other's suggestion but for a slice-of-life Yotsuba&! would not be the powerful piece it is without its backgrounds. After establishing whatever it needs it then can cut to face panels for a tad before highlighting another background. Said background usually there to establish character's placements from each other if not establishing a new mood/scene.
Heck- even in action series where its SUPER important to know character's relation to each other in an environment things like environment can forego.
Comics that are about pushing powerful environments take steps back when the scene doesn't need to be with the character. Vinland Saga has crazy details some panels and others... just faces. Heck some panels are stronger for it because details aren't getting in the way of reading the character's emotions.
Aside from establishing setting/mood is there something about the background the reader needs to understand the character's dialogue? If not, you can simplify it or subtract it.