That's a very good question and one I ask myself often, since I also have a fantasy comic with horror elements. There can be a lot of overlap between the two genres, and drawing the line is indeed difficult. However, there are a few things that I think can make a difference. Namely:
-Gore/body horror. Yes, there IS violence in fantasy too. Characters can be beheaded, lose an arm/hand/whatever, there are great battles, people die. However, depictions of the violent act itself are not usually the main focus of fantasy stories: the scenes themselves tend to be pretty quick, with not too much attention given to the gory details. If your comic is featuring multiple scenes in which violent acts are depicted in great detail, we're probably in front of a horror or at very least dark fantasy type of work. Basically: think LotR fights vs The Witcher.
-Grey morality: while fantasy usually has two clearly defined sides -the good guys and the bad guys- this distinction is not necessarily as obvious in dark fantasy or horror. Note, I'm not saying that ALL fantasy is like that: there are plenty of fantasy protagonists whose morality ain't exactly the purest (see Conan the Barbarian), though even then... there's often an even bigger evil our heroes need to fight, while dark fantasy/horror tends to make you question more often who the real bad guy is. Again, see The Witcher or Game of Thrones. Should be noted that grey morality alone isn't enough to mark a story as strictly horror, though it can definitely be a sign that we're in front of a dark fantasy.
-Dark lord vs dark fears: the bad guy of a fantasy work is usually some kind of big bad evil that is capable of putting an entire nation, world or even universe in danger. It can often be interpreted as a political metaphor, with historical or contemporary references. Heroes need to destroy the evil dark lord because it's the right thing to do. Granted, their own lives are at stake too, but once again, the fight of good vs evil tends to be the main theme. With horror works, however, the bad guy is usually something a lot smaller and a lot more personal: fear. You can have a big bad guy which puts an entire nation/world/universe in danger (e.g. a zombie attack), but the main difference is that what this bad guy usually represents is nothing else than humans' worst fears: repressed sexual desires, illnesses, losing control, temptation, murder. Stakes usually tend to feel a lot more personal in horror works than they do in fantasy: the fight against the big bad evil is a fight against your own fears.
I'm sure there are more, but these are the ones that come to mind as of now 