Getting a fanbase that makes complete asses out of themselves and make your work look bad, or makes people not wanting to look at your work because 'if it's got fans like that then it's obviously a pretty shit series'
Also, when you get those readers that completely misinterpret the story. Example "This story is about the deep roots racism has, how foolish it it, and how understanding is better than fear." but the fans read it as "YEAH WAR STORY!!! OMG I CAN'T WAIT FOR SIDE_A TO BOMB THE SHIT OUT OF SIDE_B. THE BUILD UP FOR THIS WWWWWWWWWWAAAAAAAAAAAAAAAAAARRRRRRRRRRRRRR IS HYPE!!!!!!!!!!!!!!!!!!!!!!"
Another example "This main character is bi-sexual and going through the struggles of understanding himself and his feelings." gets turns into "GAY BOIS GAY BOIS GAY BOIS GAY BOIS GAY BOI GAAAAAAAAAAAAAAAY BOOOOOOOOOOOOOOOOOOOOOOOOOISSSSSSSSSS when are they going to BANG???????"