For me, it's any time I have to draw a face looking upwards, or substantially downwards. Even with reference, it's hard.
The way I draw anatomy from the neck-down runs reasonably close to life, but the way I draw faces is more stylised. Following a reference too closely makes them look stiff.
Which I guess means I should grit my teeth, get out my charcoal and do some dang studies until said difficulty is less difficult. 