We present a navigation system using ultrasonic sensors for unknown and dynamic indoor environments. To achieve the robustness and flexibility of the mobile robot, we develop a behavior-based system architecture, consisting of multi-layered behaviors. Basic behaviors required for the navigation of a mobile robot, such as, avoiding obstacles, moving towards free space, and following targets, are redundantly developed as agents and combined in a behavior-based system architecture. An extended potential filed method is developed to produce the appropriate velocity and steering commands for the behaviors of the robot. We demonstrate the capabilities of our system through real world experiments in unstructured dynamic office environments using an indoor mobile robot.

We present a method for recognition of two-hand gestures. Two-hand gestures include fine-grain descriptions of hands under a complicated background, and have complex dynamic behaviors. Hence, assuming that two-hand gestures are an interacting process of two hands whose shapes and motions are described by switching linear dynamics, we propose a coupled switching linear dynamic model to capture interactions between both hands. The parameters of the model are learned via EM algorithm using approximate computations. Recognition is performed by selection of the model with maximum likelihood out of a few learned models during tracking. We confirmed the effectiveness of the proposed model in tracking and recognition of two-hand gestures through some experiments.

An effective human-robot interaction is essential for wide penetration of service robots into the market. Such robot needs a vision system to recognize objects. It is, however, difficult to realize vision systems that can work in various conditions. More robust techniques of object recognition and image segmentation are essential. Thus, we have proposed to use the human user's assistance for objects recognition through speech. This paper presents a system that recognizes objects in occlusion and/or multicolor cases using geometric and photometric analysis of images. Based on the analysis results, the system makes a hypothesis of the scene. Then, it asks the user for confirmation by describing the hypothesis. If the hypothesis is not correct, the system generates another hypothesis until it correctly understands the scene. Through experiments on a real mobile robot, we have confirmed the usefulness of the system.

Presently, mobile robots are navigated by means of a number of methods, using navigating systems such as the sonar-sensing system or the visual-sensing system. These systems each have their strengths and weaknesses. For example, although the visual system enables a rich input of data from the surrounding environment, allowing an accurate perception of the area, processing of the images invariably takes time. The sonar system, on the other hand, though quicker in response, is limited in terms of quality, accuracy and range of data. Therefore, any navigation methods that involves only any one system as the primary source for navigation, will result in the incompetency of the robot to navigate efficiently in a foreign, slightly-more-complicated-than-usual surrounding. Of course, this is not acceptable if robots are to work harmoniously with humans in a normal office/laboratory environment. Thus, to fully utilise the strengths of both the sonar and visual sensing systems, this paper proposes a fusion of navigating methods involving both the sonar and visual systems as primary sources to produce a fast, efficient and reliable obstacle-avoiding and navigating system. Furthermore, to further enhance a better perception of the surroundings and to improve the navigation capabilities of the mobile robot, active sensing modules are also included. The result is an active sensor fusion system for the collision avoiding behaviour of mobile robots. This behaviour can then be incorporated into other purposive behaviours (eg. Goal Seeking, Path Finding, etc. ). The validity of this system is also shown in real robot experiments.

We are developing a helper robot that carries out tasks ordered by the user through speech. The robot needs a vision system to recognize the objects appearing in the orders. It is, however, difficult to realize vision systems that can work in various conditions. Thus, we have proposed to use the human user's assistance through speech. When the vision system cannot achieve a task, the robot makes a speech to the user so that the natural response by the user can give helpful information for its vision system. Our previous system assumes that it can segment images without failure. However, if there are occluded objects and/or objects composed of multicolor parts, segmentation failures cannot be avoided. This paper presents an extended system that tries to recover from segmentation failures using photometric invariance. If the system is not sure about segmentation results, the system asks the user by appropriate expressions depending on the invariant values. Experimental results show the usefulness of the system.

We present a method to track and recognize shape-changing hand gestures simultaneously. The switching linear model using active contour model well corresponds to temporal shapes and motions of hands. However, inference in the switching linear model is computationally intractable, and therefore the learning process cannot be performed via the exact EM (Expectation Maximization) algorithm. Thus, we present an approximate EM algorithm using a collapsing method in which some Gaussians are merged into a single Gaussian. Tracking is performed through the forward algorithm based on Kalman filtering and the collapsing method. We also present a regularized smoothing, which plays a role of reducing jump changes between the training sequences of shape vectors representing complex-variable hand shapes. The recognition process is performed by the selection of a model with the maximum likelihood from some trained models while tracking is being performed. Experiments for several shape-changing hand gestures are demonstrated.

This paper proposes an integrated approach to simultaneous detection and localization of multiple object categories using both generative and discriminative models. Our approach consists of first generating a set of hypotheses for each object category using a generative model (pLSA) with a bag of visual words representing each object. Based on the variation of objects within a category, the pLSA model automatically fits to an optimal number of topics. Then, the discriminative part verifies each hypothesis using a multi-class SVM classifier with merging features that combines spatial shape and appearance of an object. In the post-processing stage, environmental context information along with the probabilistic output of the SVM classifier is used to improve the overall performance of the system. Our integrated approach with merging features and context information allows reliable detection and localization of various object categories in the same image. The performance of the proposed framework is evaluated on the various standards (MIT-CSAIL, UIUC, TUD etc.) and the authors' own datasets. In experiments we achieved superior results to some state of the art methods over a number of standard datasets. An extensive experimental evaluation on up to ten diverse object categories over thousands of images demonstrates that our system works for detecting and localizing multiple objects within an image in the presence of cluttered background, substantial occlusion, and significant scale changes.

This paper presents a vision-based human interface system that enables a user to move a target object in a 3D CG world by moving his hand. The system can interpret hand motions both in a frame fixed in the world and a frame attached to the user. If the latter is chosen, the user can move the object forward by moving his hand forward even if he has changed his body position. In addition, the user does not have to keep in mind that his hand is in the camera field of view. The active camera system tracks the user to keep him in its field of view. Moreover, the system does not need any camera calibration. The key for the realization of the system with such features is vision algorithms based on the multiple view affine invariance theory. We demon-strate an experimental system as well as the vision algorithms. Human operation experiments show the usefulness of the system.

This introduces a method which uses LIDAR to identify humans and track their positions, body orientation, and movement trajectories in any public space to read their various types of behavioral responses to surroundings. We use a network of LIDAR poles, installed at the shoulder level of typical adults to reduce potential occlusion between persons and/or objects even in large-scale social environments. With this arrangement, a simple but effective human tracking method is proposed that works by combining multiple sensors' data so that large-scale areas can be covered. The effectiveness of this method is evaluated in an art gallery of a real museum. The result revealed good tracking performance and provided valuable behavioral information related to the art gallery.

Conventional interest point based matching requires computationally expensive patch preprocessing and is not appropriate for recognition of plain objects with negligible detail. This paper presents a method for extracting distinctive interest regions from images that can be used to perform reliable matching between different views of plain objects or scene. We formulate the correspondence problem in a Naive Bayesian classification framework and a simple correlation based matching, which makes our system fast, simple, efficient, and robust. To facilitate the matching using a very small number of interest regions, we also propose a method to reduce the search area inside a test scene. Using this method, it is possible to robustly identify objects among clutter and occlusion while achieving near real-time performance. Our system performs remarkably well on plain objects where some state-of-the art methods fail. Since our system is particularly suitable for the recognition of plain object, we refer to it as Simple Plane Object Recognizer (SPOR).

The detection of object categories with large variations in appearance is a fundamental problem in computer vision. The appearance of object categories can change due to intra-class variations, background clutter, and changes in viewpoint and illumination. For object categories with large appearance changes, some kind of sub-categorization based approach is necessary. This paper proposes a sub-category optimization approach that automatically divides an object category into an appropriate number of sub-categories based on appearance variations. Instead of using predefined intra-category sub-categorization based on domain knowledge or validation datasets, we divide the sample space by unsupervised clustering using discriminative image features. We then use a cluster performance analysis (CPA) algorithm to verify the performance of the unsupervised approach. The CPA algorithm uses two performance metrics to determine the optimal number of sub-categories per object category. Furthermore, we employ the optimal sub-category representation as the basis and a supervised multi-category detection system with χ2 merging kernel function to efficiently detect and localize object categories within an image. Extensive experimental results are shown using a standard and the authors' own databases. The comparison results reveal that our approach outperforms the state-of-the-art methods.

Service robots need to be able to recognize and identify objects located within complex backgrounds. Since no single method may work in every situation, several methods need to be combined and robots have to select the appropriate one automatically. In this paper we propose a scheme to classify situations depending on the characteristics of the object of interest and user demand. We classify situations into four groups and employ different techniques for each. We use Scale-invariant feature transform (SIFT), Kernel Principal Components Analysis (KPCA) in conjunction with Support Vector Machine (SVM) using intensity, color, and Gabor features for five object categories. We show that the use of appropriate features is important for the use of KPCA and SVM based techniques on different kinds of objects. Through experiments we show that by using our categorization scheme a service robot can select an appropriate feature and method, and considerably improve its recognition performance. Yet, recognition is not perfect. Thus, we propose to combine the autonomous method with an interactive method that allows the robot to recognize the user request for a specific object and class when the robot fails to recognize the object. We also propose an interactive way to update the object model that is used to recognize an object upon failure in conjunction with the user's feedback.

Eye contact is an effective means of controlling human communication, such as in starting communication. It seems that we can make eye contact if we simply look at each other. However, this alone does not establish eye contact. Both parties also need to be aware of being watched by the other. We propose a method of bidirectional eye contact satisfying these conditions for human-robot communication. When a human wants to start communication with a robot, he/she watches the robot. If it finds a human looking at it, the robot turns to him/her, changing its facial expressions to let him/her know its awareness of his/her gaze. When the robot wants to initiate communication with a particular person, it moves its body and face toward him/her and changes its facial expressions to make the person notice its gaze. We show several experimental results to prove the effectiveness of this method. Moreover, we present a robot that can recognize hand gestures after making eye contact with the human to show the usefulness of eye contact as a means of controlling communication.