Autonomous driving is not only required to detect pedestrians around vehicles, but also expected to understand the behaviors of pedestrians. Pedestrian body orientation and head orientation are the relevant indicators of the pedestrian intention. This paper proposes an accurate estimation system to recognize the pedestrian body orientation and the pedestrian head orientation from on-board camera and inertial sensors. The proposed system discretizes the body orientation and the head orientation into 16 directions. In order to achieve the accurate orientation estimation, a novel training database is established, which includes strongly labeled data and weakly labeled data. Semi-Supervised Learning method is employed to annotate the weakly labeled data, and to generate the accurate classifier based on the proposed training database. In addition, the temporal constraint and the human physical model constraint are considered in orientation estimation, which are beneficial to the reasonable and stable result of orientation estimation for the pedestrian in image sequences. This estimated result is the orientation in camera space. The comprehension of the pedestrian behavior needs to be conducted in the real world space. Therefore, this paper proposes to model the motion of the host vehicle using inertial sensor, then transforms the estimated orientation from camera space to the real world space by considering the vehicle and pedestrian motion. The represented orientation indicates the behavior of the pedestrian more directly. Finally, a series of experiments demonstrate the effectiveness of the proposed pedestrian orientation system.

This paper proposes a sound source orientation estimation method that is suitable for a distributed microphone arrangement. The proposed method is based on orientation-extended beamforming (OEBF), which has four features: (a) robustness against reverberations, (b) robustness against noises, (c) free arrangements of microphones and (d) feasibility for real-time processing. In terms of (a) and (c), since OEBF is based on a general propagation model using transfer functions (TFs) that include all propagation phenomena such as reflections and diffractions, OEBF causes no model errors for the propagation phenomena, and is applicable to arbitrary microphone arrangements. Regarding (b), OEBF overcomes noise effects by incorporating three additional processes (Amplitude extraction, time-frequency mask and histogram integration) that are also proposed in this paper. As for (d), OEBF is executable in real-time basis as the execution process is the same as usual beamforming processes. A numerical experiment was performed to confirm the theoretical validity of OEBF. The results showed that OEBF was able to estimate sound source positions and orientations very precisely. Practical experiments were carried out using a 96-channel microphone array in real environments. The results indicated that OEBF worked properly even under reverberant and noisy environments and the averaged estimation error was given only 4°.

In this work, spatial information consisting of the position and orientation angle of an acoustic source is estimated by an artificial neural network (ANN). The estimated position of a speaker in an enclosed space is used to refine the estimated time delays for a delay-and-sum beamformer, thus enhancing the output signal. On the other hand, the orientation angle is used to restrict the lexicon used in the recognition phase, assuming that the speaker faces a particular direction while speaking. To compensate the effect of the transmission channel inside a short frame analysis window, a new cepstral mean normalization (CMN) method based on a Gaussian mixture model (GMM) is investigated and shows better performance than the conventional CMN for short utterances. The performance of the proposed method is evaluated through Japanese digit/command recognition experiments.

In this letter, we present a real-time orientation estimation and motion tracking scheme using interacting multiple model (IMM) based Kalman filtering method. Two nonlinear filters, quaternion-based extended Kalman filter (QBEKF) and gyroscope-based extended Kalman filter (GBEKF) are utilized in the proposed IMM-based orientation estimator for sensor motion state estimation. In the QBEKF, measurements from gyroscope, accelerometer and magnetometer are processed; while in the GBEKF, sole measurements from gyroscope are processed. The interacting multiple model algorithm is used for fusing the estimated states via adaptive model weighting. Simulation results validate the proposed design concept, and the scheme is capable of reducing overall estimation errors in sensor motion tracking.

This paper proposes a new emphasizing three-dimensional pointing device considering user friendliness and lack of cable clutter. The proposed method utilizes five degrees of freedom via the medium of non-verbal voice of human. That is, the spatial direction of the sound source, the type of the voice phoneme and the tone of the voice phoneme are utilized. The input voice is analyzed regarding the above factors and then taking proper effects as previously defined for human interface. In this paper the estimated spatial direction is used for three-dimensional movement for the virtual object as three degrees of freedom. Both of the type and the tone of the voice phoneme are used for remaining two degrees of freedom. Since vocalization of nonverbal human voice is an everyday task, and the intonation of the voice can be quite easily and intentionally controlled by human vocal ability, the proposed scheme is a new three-dimensional spatial interaction medium. In this sense, this paper realizes a cost-effective and handy nonverbal interface scheme without any artificial wearing materials which might give a physical and psychological fatigue. By using the prototype the authors evaluate the performance of the scheme from both of static and dynamic points of view and show some advantages of look and feel, and then prospect possibilities of the application for the proposed scheme.