People are being inundated under enormous volumes of information and they often dither about making the right choices from these. Interactive user support by information service system such as concierge services will effectively assist such people. However, human-machine interaction still lacks naturalness and thoughtfulness despite the widespread utilization of intelligent systems. The system needs to estimate user's interest to improve the interaction and support the choices. We propose a novel approach to estimating the interest, which is based on the relationship between the dynamics of user's eye movements, i.e., the endogenous control mode of saccades, and machine's proactive presentations of visual contents. Under a specially-designed presentation phase to make the user express the endogenous saccades, we analyzed the timing structures between the saccades and the presentation events. We defined resistance as a novel time-delay feature representing the duration a user's gaze remains fixed on the previously presented content regardless of the next event. In experimental results obtained from 10 subjects, we confirmed that resistance is a good indicator for estimating the interest of most subjects (75% success in 28 experiments on 7 subjects). This demonstrated a higher accuracy than conventional estimates of interest based on gaze duration or frequency.

This paper addresses the parameter estimation problem of an interval-based hybrid dynamical system (interval system). The interval system has a two-layer architecture that comprises a finite state automaton and multiple linear dynamical systems. The automaton controls the activation timing of the dynamical systems based on a stochastic transition model between intervals. Thus, the interval system can generate and analyze complex multivariate sequences that consist of temporal regimes of dynamic primitives. Although the interval system is a powerful model to represent human behaviors such as gestures and facial expressions, the learning process has a paradoxical nature: temporal segmentation of primitives and identification of constituent dynamical systems need to be solved simultaneously. To overcome this problem, we propose a multiphase parameter estimation method that consists of a bottom-up clustering phase of linear dynamical systems and a refinement phase of all the system parameters. Experimental results show the method can organize hidden dynamical systems behind the training data and refine the system parameters successfully.

This paper describes a distinctive phonetic feature (DPF) extraction method for use in a phoneme recognition system; our method has a low computation cost. This method comprises three stages. The first stage uses two multilayer neural networks (MLNs): MLNLF-DPF, which maps continuous acoustic features, or local features (LFs), onto discrete DPF features, and MLNDyn, which constrains the DPF context at the phoneme boundaries. The second stage incorporates inhibition/enhancement (In/En) functionalities to discriminate whether the DPF dynamic patterns of trajectories are convex or concave, where convex patterns are enhanced and concave patterns are inhibited. The third stage decorrelates the DPF vectors using the Gram-Schmidt orthogonalization procedure before feeding them into a hidden Markov model (HMM)-based classifier. In an experiment on Japanese Newspaper Article Sentences (JNAS) utterances, the proposed feature extractor, which incorporates two MLNs and an In/En network, was found to provide a higher phoneme correct rate with fewer mixture components in the HMMs.

Studies on gaze analysis have revealed some of the relationships between viewers' gaze and their internal states (e.g., interests and intentions). However, understanding content browsing behavior in uncontrolled environments is still challenging because human gaze can be very complex; it is affected not only by viewers' states but also by the spatio-semantic structures of visual content. This study proposes a novel gaze analysis framework which introduces the content creators' point of view to understand the meaning of browsing behavior. Visual content such as web pages, digital articles and catalogs are comprised of structures intentionally designed by content creators, which we refer to as designed structure. This paper focuses on two design factors of designed structure: spatial structure of content elements (content layout), and their relationships such as “being in the same group”. The framework was evaluated with an experiment involving 12 participants, wherein the participant's state was estimated from their gaze behavior. The results from the experiment show that the use of design structure improved estimation accuracies of user states compared to other baseline methods.