I think about the same thing. Not in the way were you make a conlang for a separate civilization, but in the way that your main story takes place in the civilization you make a conlang for. My story takes place in a fictional civilization with a totally different culture and different practices, and for certain phenomenons in it, there just aren't any English words. So I'm not making an entirely different language, just inserting words here and there.
Anyway, pros and cons. For my purposes, the pro is that you're not gonna have to use a replacement word from a different culture. Like, in my novel the word "Akaneb" means a upper middle caste religious person. If I was to use a real-life word substitute for this idea. For instance, the Indian caste system word "Brahmin" which might be more understandable or familiar to some people there would be two effects. First of all, I am a white woman and using an Indian word for my own Exotic! and Foriegn! civilization would be suuuuper disrespectful and just like, wrong on so many levels. The other effect would be that maybe now my reader automatically assumes my fake civilization has an Indian aesthetic and Indian values, when really I have based my civilization on ancient Egyptian and other ancient Mesopotamian civilizations.
The con is that it can be really confusing and tedious for the reader. Like, if you're gonna use it in the same way Tolkien uses elvish, I think it's fine because you're seeing the story from the hobbit perspective, and the hobbits don't understand elvish so the reader feels no need to learn any elvish words either. However, if you use it in the way I use it, it can be confusing. Especially if you're forgetful or not a super engaged reader. And you don't want to keep shoving the meaning of the word in your reader's face over and over again, because info dumps are annoying and in a way it feels like you're insulting the reader's intelligence.
Anyway when I was doing all my world-building for my fictional world I decided the pros outweighed the cost, and I insert a few fake words here and there. There are a few things, however, that I have given my own words, but I use regular English words for anyway just to prevent it getting too complicated (for instance, I made up my own military ranks but in the story I just use regular words like captain, general, commander. Lol.)