My backgrounds I use photo references for often, but I design them so the line and patterns of dark/light serve the overall page. I think composition first, what kind of shot might be interesting to draw or look and what's going to serve the page overall. They can create a sense of pacing as well as scale, and help you create a strong sense of place. Here's a self plug from my comic because I'm vain and think this is "money where my mouth is". This background is minimal but helps guide the page, and you can tell it's a forest even if the colours aren't perfectly realistic. There's no point in having ultra detailed photorealistic backgrounds that clutter up a page or add nothing. Also - not every panel has to have 'em! It can feel cluttered if every single little panel has some background element. Let yourself breathe sometimes.
I also like backgrounds that tell a story about the characters or the place they're in.
As for time - a long time! Not as long as characters, but that's because I tend to unify my colours in the background and keep it to one or two colours + my pooling blacks. But, I think about them from the start so I never really divide myself into thinking about "backgrounds" and "not backgrounds" it's all just "lines" to me. But you can fudge a background. I let mine fade into a black and don't render everything and that's my secret to making pages in a reasonable time.
