You are not alone on this but
there are some simple things
that you can do which will make
all of your face drawings 300% better:
-don´t rush, it´s not a race
-do a sketch before you do the final outline
-start with a circle for the upper part of the skull, attach a chin instead of drawing everything into an oval shape
-draw a line at the center of the circle, depending on which direction the person is looking
-imagine that the character is wearing glasses, it makes it easier to allign the nose, eyes and ears, you can also draw the glasses in the sketch
It doesn´t have to be a perfect circle, circles get better with practice
and I also wouldn´t stress about it in the sketch phase of the drawing.
My face drawings and drawings in general sucked big time but got better
through practice, not rushing, sketching. It took time to see results
Here is a simplified way to draw the head with the circle and the attached chin and what I mean
with imagining or drawing the glasses.