In this paper, we first discuss on a framework for a 3D image display system which is the combination of passive sensing and active display technologies. The passive sensing enables to capture real scenes under natural condition. The active display enables to present arbitrary views with proper motion parallax following the observer's motion. The requirements of passive sensing technology for 3D image displays are discussed in comparison with those for robot vision. Then, a new stereo algorithm, called SEA (Stereo by Eye Array), which satisfies the requirements is described in detail. The SEA uses nine images captured by a 33 camera array. It has the following features for depth estimation: 1) Pixel-based correspondence search enables to obtain a dense and high-spatial-resolution depth map. 2) Correspondence ambiguity for linear edges with the orientation parallel to a particular baseline is eliminated by using multiple baselines with different orientations. 3) Occlusion can be easily detected and an occlusion-free depth map with sharp object boundaries is generated. The feasibility of the SEA is demonstrated by experiments by using real image data.

Facial micro-expressions are fast and subtle facial motions that are considered as one of the most useful external signs for detecting hidden emotional changes in a person. However, they are not easy to detect and measure as they appear only for a short time, with small muscle contraction in the facial areas where salient features are not available. We propose a new computer vision method for detecting and measuring timing characteristics of facial micro-expressions. The core of this method is based on a descriptor that combines pre-processing masks, histograms and concatenation of spatial-temporal gradient vectors. Presented 3D gradient histogram descriptor is able to detect and measure the timing characteristics of the fast and subtle changes of the facial skin surface. This method is specifically designed for analysis of videos recorded using a hi-speed 200 fps camera. Final classification of micro expressions is done by using a k-mean classifier and a voting procedure. The Facial Action Coding System was utilized to annotate the appearance and dynamics of the expressions in our new hi-speed micro-expressions video database. The efficiency of the proposed approach was validated using our new hi-speed video database.

Visualizing occluded objects is a useful applications of Mixed Reality (MR), which we call "see-through vision." For this application, it is important to display occluded objects in such a manner that they can be recognized intuitively by the user. Here, we evaluated four visualization methods for see-through vision that can aid the user to recognize occluded objects in outdoor scenes intuitively: "elimination of occluding objects," "ground grid," "overlaying model of occluding object," and "top-down view." As we used a new handheld MR device for outdoor see-through vision, we performed subjective experiments to determine the best combination of methods. The experimental results indicated that a combination of showing the ground grid, overlaying wireframe models of occluding objects, and top-down view to be optimal, while it was not necessary to display occluding objects for outdoor see-through vision with a handheld device, because users can see them with the naked eye.

In order to realize a robust computer vision system which can cope with a variety of scenes, we propose a new scheme to integrate multiple vision algorithms in a cooperative framework. The key concepts of the scheme can be summarized as follows: (1) A vision algorithm is implemented as an independent specialist. (2) There are multiple specialists in the system. (3) All specialists work in parallel. (4) Each specialist tries to solve the vision problem in its own way. (5) A specialist may communicate with another specialist when it needs a help. The scheme clearly has a nature of task parallelism and is suitable to be built on a parallel computer. In order to demonstrate the feasibility of the proposed scheme, we are developing two computer vision systems in two different approaches which are both based on the same concept. They are called a cooperative approach and a concurrent top-down approach. Roughly speaking, the former is an integration of multiple algorithms in a bottom-up framework while the latter is in a top-down framework. Preliminary results obtained by the systems are also presented.