I've really only done one scary looking monster that came out looking good, and what I did was look at animal photos as a reference.
In this instance my neighbor was doing a school project where he needed a scene from a book. The kids could have someone else draw it, so he asked me. He asked for "a crocodile-snake monster about to eat a girls face off" so I did it. But I had no idea how to draw crocodiles so I looked at a lot of pictures. I also wanted it to look big and muscular so I tried to make it bulky.
It's quite old but at the time when I drew it it was actually one of the best things I'd ever drawn.
Of course I digitally inked it and my neighbor colored it. Anyway, as you can see it's not perfect, but it was my first time doing something scary.
Another thing I heard was the creator of Melvina's Therapy (on Webtoons), which is an amazing horror comic I totally recommend, said that for the first story's monster they used images of MRI scans of the human face - which look very creepy. Not only that but they also used animals as a reference to create some disturbing body horror.
Another thing you can try is to take the cute monsters and make them so cute they become unsettling and creepy. Franken Fran, one of my favorite horror manga (chalk full of medical body horror) has a chapter where this girl wants to look more like an anime character and undergoes such a dramatic series of surgeries that her body can hardly function... then as she's about to get it on with this guy, she looks like an alien. I believe it's Chapter 37 if you're interested in it. Actually the whole manga is full of "seemingly cute girls who end up as monsters". It's good if you want a body horror thing.
That said I have the same issue, my monsters are always too cute. So this is all the advice I can really give. lol