You have a lot segments that you dont need. The best way to avoid running into this problem is by laying all your important loops down first. Look at the picture below, and see how he lays all the important loops down. The eyes, mouth, jaw, base of the skull etc. you will cleaner topology which is more important than the overall shape of the mesh itself, especially when it comes to animation.
Another way you could do it is by sculpting out the head in zbrush then retopologizing it later. Which every way you feel comfortable with. Hope I helped... Good Luck