Pronunciation is a fundamental factor in speaking and listening. However, instructions for important articulation have not been sufficiently provided in conventional computer-assisted language learning (CALL) systems. One typical case is the articulation of rounded vowels. Although lip protrusion is essential for their correct pronunciation, the perception of lip protrusion is often difficult for beginners. To tackle this issue, we propose an innovative method that will provide a comprehensive visual explanation for articulation. Lip movements are three-dimensionally measured, and face images or videos are pseudocoloured on the basis of the movements. The coloured regions represent the lip protrusion of rounded vowels. To verify the learning effect of the proposed method, we conducted experiments with Japanese undergraduates in Chinese classes. The results showed that our method has advantages over conventional video materials.

Self-paced e-learning provides much more freedom in time and locale than traditional education as well as diversity of learning contents and learning media and tools. However, its limitations must not be ignored. Lack of information on learners' states is a serious issue that can lead to severe problems, such as low learning efficiency, motivation loss, and even dropping out of e-learning. We have designed a novel e-learning support system that can visually observe learners' non-verbal behaviors and estimate their learning states and that can be easily integrated into practical e-learning environments. Three pairs of internal states closely related to learning performance, concentration-distraction, difficulty-ease, and interest-boredom, were selected as targets of recognition. In addition, we investigated the practical problem of estimating the learning states of a new learner whose characteristics are not known in advance. Experimental results show the potential of our system.

This article introduces our investigation on learning state estimation in e-learning on the condition that visual observation and recording of a learner's behaviors is possible. In this research, we examined methods of adaptation for a new learner for whom a small number of ground truth data can be obtained.