We propose a new method to precisely detect pupil contours in face images. Pupil contour detection is necessary for various applications using face images. It is, however, difficult to detect pupils precisely because of their weak edges or lack of edges. The proposed method is based on minimizing the energy of pattern and edge. The basic idea of this method is that the energy, which consists of the pattern and the edge energy, has to be minimized. An efficient search method is also introduced to overcome the underlying problem of efficiency in energy minimization methods. "Guide patterns" are introduced for this purpose. Moreover, to detect pupils more precisely we use an ellipse model as pupil shape in this paper. Experimental results show the effectiveness of the proposed method.