Self-paced e-learning provides much more freedom in time and locale than traditional education as well as diversity of learning contents and learning media and tools. However, its limitations must not be ignored. Lack of information on learners' states is a serious issue that can lead to severe problems, such as low learning efficiency, motivation loss, and even dropping out of e-learning. We have designed a novel e-learning support system that can visually observe learners' non-verbal behaviors and estimate their learning states and that can be easily integrated into practical e-learning environments. Three pairs of internal states closely related to learning performance, concentration-distraction, difficulty-ease, and interest-boredom, were selected as targets of recognition. In addition, we investigated the practical problem of estimating the learning states of a new learner whose characteristics are not known in advance. Experimental results show the potential of our system.