Backgrounds take a lot of time but even so, when I'm planning a page, I still end up sketching them or straight-up dedicating some panels to establishing an environment whenever. They're most important during silent pages when a character is exploring a space or when characters are walking and talking; stuff like that. (This is on-top of what everyone else has already succinctly described). Then when I have to actually draw the final version, I curse myself for putting myself through it. But after I'm done, I'm pretty satisfied.
As far as which panels should have a background or which not... if I have already established where they are, I will usually not bother cuz time is money and my main focus is often characters' expressions and gestures. When there's some long dialogue, I'll use that to break the monotony of "floating heads" or busts, I'll draw a bigger pic of the space they're in and lead the reader through it with the dialogue bubbles or w/e.
Like @Michelle I really like seeing the characters interact with their environment and props. Managing to do get that feeling of "they're really in that space" in my own comic is something I strive for. And it's fun.