This article about schizophrenia is a really good example of the kind of thing I'm talking about! It's true that schizophrenia can include hallucinations, but our ideas of what it's like to experience hallucinations or hear voices mostly come from pop culture -- so getting perspective from an actual person who deals with this on a regular basis talking about her experiences is super important to understanding how to portray it without leaning on stereotypes -- or, frankly, deciding if it's a good idea to use that element for this character at all.
I'll second the idea that he doesn't have to be mentally ill to have bad or weird behaviour like this. You can just make him violent with an over-the-top temper, laughing at inappropriate times, and not worry about making him "crazy."
I'll add to this, as well -- think about whether this is using mental illness to make them creepy. Does giving this character a mental illness actually enhance the character, or is it just an easy way to make the character seem "off"? If it's that second one, it might not be the best idea.
I can't help but notice that mentally ill people who've posted in this thread talk about how the illness affects the mentally ill person's life ("likely they regret it afterwards and hate themselves for doing it. Likely they ruin friendships, relationships and stuck alone fearing when they'll fall into mania again") while you describe the effects this mental illness would have on the people around him ("The illness is just to convey that everything is not alright, because he will have violent outbursts")
There's nothing on how it affects him, how it affects his life and character. Like, a character who experiences hallucinations might need to keep someone else around to help them out when they're not sure if something's real, and being dependent on others in that way is tough! An OCD character might have rituals they're embarrassed and frustrated by but have to do, or things they can't handle because of the anxiety it'd cause. My DID/MPD friend kept a checklist of even simple things like "did I eat lunch already" because one "personality" might not know what the other had taken care of.
It can't just be "how does this make him come across to others" but "how does this affect him, and his life, and the things he's learned to do, or needs help to do, or has more experience dealing with than other people?"
I feel like this is what's making this difficult. You're starting from a base that's very hard to alter -- the character Has To Be Like This Because He Just Is Evil -- and then asking "so, how do I make this character realistically mentally ill?" If you REALLY want to do that in a thoughtful way, you might have to be prepared to rethink some aspects of your character.
Honestly, the character you are describing can't come across as realistically human... because humans aren't "just evil." You can say "he's not evil because of hearing voices, he just IS evil" but if that's all we have, we just end up with "the evil character hears voices. Of course he does."
(Note: There is one easy way to make it work while being thoughtful: if you had a main character who was also mentally ill. Like, if the vampire is schizophrenic and handling it badly, but the main character is also schizophrenic and handling it well, so the vampire isn't the only major example of schizophrenia we see, that makes a significant difference! I know that's unlikely to work for many stories, but I figured it's worth mentioning!)
TL;DR:
The character you're describing sounds to me like it would be easiest to not worry about making him mentally ill -- he can be violent and unreasonable and evil without that. He can, I dunno, hear the voices of ancient vampires or some kind of magical thing like that, without being mentally ill.
If you really want mental illness to be a part of the character, AND you want to do it in a way that's thoughtful to mentally ill people, I think it's gonna be hard to incorporate both of those things while working with a character who Just Is Evil!
I'd say, go ahead and read up on the mental illnesses you've been considering! Read people talking about what it's like to live with that mental illness, and see if you can imagine your vampire having those experiences. Think about whether it's something you can incorporate into his character, or if those feelings and experiences wouldn't fit "because he's evil" -- because if it's that second one, the mental illness might not be a good fit for the character you want to portray!