A mixed of references and feeling. There's usually a vibe I want my character to have, so I design around that. I also look up pictures that describe the personality, culture, or setting my characters are in and use that to help me.
I'm also learning to love color coordinating my cast. I'll pick a color and have the group designed around it. Also helps when I want my MC to really stand out!