Surface reconstruction and visualization from sparse and incomplete surface data is a fundamental problem and has received growing attention in both computer vision and graphics. This paper presents a computational scheme for realistic visualization of free-formed surfaces from 3D range images. The novelty of this scheme is that by integrating computer vision and computer graphics techniques, we dynamically construct a mesh representation of the arbitrary view of the surfaces, from a view-invariant shape description obtained from 3D range images. We outline the principle of this scheme and describle the frame work of a graphical reconstruction model, we call arbitrarily oriented meshes', which is developed based on differential geometry. The experimental results on real range data of human faces are shown.

This paper describes the results of tests that measured the allowable delay between images and tactile information via a force feedback device. In order to investigate the allowable delay, two experiments were performed: 1) subjective evaluation in real space and 2) subjective evaluation in virtual space using a force feedback device.

In a display system with a line-of-gaze (LOG) controller, it is difficult to make the directions and motions of a LOG-controlled object coincide as closely as possible in the display with the user's intended LOG-directions and motions. This is because LOG behavior is not only smooth, but also saccadic due to the problem of involuntary eye movement. This article introduces a flexible on-line LOG-control scheme to realize nearly perfect LOG operation. Using a mesh-wise cursor pattern, the first visual experiment elucidates subjectively that a Kalman Filter (KF) for smoothing and predicting is effective in filtering out macro-saccadic changes of the LOG and in predicting sudden changes of the saccade while movement is in progress. It must be assumed that the LOG trajectory can be described by a linear position-velocity-acceleration approximation of Sklansky Model (SM). Furthermore, the second experiment uses a four-point pattern and simulations to scrutinize the two physical properties of velocity and direction-changes of the LOG in order to quantitatively and efficiently resolve "moving" and "gazing". In order to greatly reduce the number of LOG-small-position changes while gazing, the proposed Gaze-Holding algorithm (GH) with a gaze-potential function is combined with the KF. This algorithm allows the occurrence frequency of the micro-saccade to be reduced from approximately 25 Hz to 1 or 2 Hz. This great reduction in the frequency of the LOG-controlled object moves is necessary to achieve the user's desired LOG-response while gazing. Almost perfect LOG control is accomplished by the on-line SM+KF+GH scheme while either gazing or moving. A menu-selection task was conducted to verify the effectiveness of the proposed on-line LOG-control method.

This paper discussed a distributed processing architecture for a virtual space teleconferencing system. Virtual space teleconferencing is a promising application field for networked virtual environments. People in different places will be able to meet each other in a virtual teleconferencing room and proceed with various cooperative tasks. When such a system creates a realistic virtual environment, it can be referred to as a "Teleconferencing with realistic sensations" system. Further more, as the conference environment can be shared by a number of users, it is possible to perform various kinds of cooperative work using the system. In this paper, the architecture for networked multi-user virtual space systems are classified, and then a case study is described for building a proposed teleconferencing system. The system reproduces a 3D image of each conference participant in a virtual meeting room. Compared with the former system, the new system can deal with more than three participants at the same time and can connect them through commercial telephone lines. Based on the virtual world database management structure, the system was classified as a central server system. However, a central server architecture limits the number of conference sites. We confirmed that the system can serve up to 14 sites using multi-modal interaction without significant latency in operation from summational experiments. Then, introducing some assumptions to the results, we have proposed processing model of the system. The results of model could describe the experimental results and we could indicate roughy estimated system capacity to realize a requaied system performance.

This paper focuses on Mixed Reality (MR) visual displays, a particular subset of Virtual Reality (VR) related technologies that involve the merging of real and virtual worlds somewhere along the virtuality continuum" which connects completely real environments to completely virtual ones. Probably the best known of these is Augmented Reality (AR), which refers to all cases in which the display of an otherwise real environment is augmented by means of virtual (computer graphic) objects. The converse case on the virtuality continuum is therefore Augmented Virtuality (AV). Six classes of hybrid MR display environments are identified. However, an attempt to distinguish these classes on the basis of whether they are primarily video or computer graphics based, whether the real world is viewed directly or via some electronic display medium, whether the viewer is intended to feel part of the world or on the outside looking in, and whether or not the scale of the display is intended to map orthoscopically onto the real world leads to quite different groupings among the six identified classes, thereby demonstrating the need for an efficient taxonomy, or classification framework, according to which essential differences can be identified. The obvious' distinction between the terms real" and virtual" is shown to have a number of different aspects, depending on whether one is dealing with real or virtual objects, real or virtual images, and direct or non-direct viewing of these. An (approximately) three dimensional taxonomy is proposed, comprising the following dimensions: Extent of World Knowledge (how much do we know about the world being displayed?"), Reproduction Fidelity (how realistically' are we able to display it?"), and Extent of Presence Metaphor (what is the extent of the illusion that the observer is present within that world?").

Recently, studies aiming at the next generation of visual communication services which support better human communication have been carried out intensively in Japan. The principal motive of these studies is to develop new services which are not restricted to a conventional communication framework based on the transmission of waveform signals. This paper focuses on three important key words in these studies; "intelligent," "real," and "distributed and collaborative," and describes recent research activities. The first key word "intelligent" relates to intelligent image coding. As a particular example, model-based coding of moving facial images is discussed in detail. In this method, shape change and motion of the human face is described by a small number of parameters. This feature leads to the development of new applications such as very low bit-rate transmission of moving facial images, analysis and synthesis of facial expression, human interfaces, and so on. The second key word "real" relates to communication with realistic sensations and virtual space teleconferencing. Among various component technologies, real-time reproduction of 3-D human images and a cooperative work environment with virtual space are discussed in detail. The last key word "distributed and collaborative" relates to collaborative work in a distributed work environment. The importance of visual media in collaborative work, a concept of CSCW, and requirements for realizing a distributed collaborative environment are discussed. Then, four examples of CSCW systems are briefly outlined.

This paper addresses thecurrent research and recent trends of image communications research that fall into the theme of intelligent communications in two parts --Intelligent Image Coding and Communications with Realistic Sensations. In Part , the theme of intelligent image coding and the particular example of encoding of human facial images using an intelligence-based approach is presented. A classification of the different generations--five generations--of image coding systems is first given. First and second generations are the direct wave-form coding and the statistical encoding methods. The third and fourth generations are the use of model-based anaylsis-synthesis and recognition/reconstruction methods, respectively. The fifth generation is the full intelligent coding system that can understand the semantics, intention, motives and other factors in an image. The progress in the model-based analysis-synthesis approach to image coding is reviewed and the particular example of its application to the encoding of human facial images is examined. In this example, the methods of modeling human face, the extraction of feature points and the tracing of motion in a facial image, as well as the analysis and synthesis of facial expressions are discussed. In Part , the current research in the new paradigm of telecommunication--Communications with Realistic Sensations is introduced. The progress in the research of a Virtual Space Teleconference System as an example of this new paradigm is presented. The possibility of delivering a very high level of sensation realism to a receiver is made real by recent advances in computer graphics and computer animation techniques, in stereoscopic display techniques that enable life-size images to be displayed on a wide screen and in computer human interface technologies, particularly, for recording human hand, head and eye positions. Progress of these technologies in the context of the research on the Virtual Space Teleconference System are discussed. The research on how to communicate and manipulate 3-D objects in a such an environment is examined.

A representation scheme is presented for shapes of three-dimensional curved objects whose surface curvatures are not continuous at all points, i.e., surfaces with C1 continuity only. The representation consists of a subset of lines of curvature and their intersections, called surface structure descriptors, organized in a three-layer hierarchy of increasing stringency. From these descriptors, a hierarchical parameterization of C1 surfaces leading to a global shape representation of the object is obtained in terms of enclosing boundaries of bumps and dents, ridge and valley lines, and peaks and pits (curvature extreme points). To handle the discontinuities of surface curvatures, we introduce a computational scheme in which gradient directions of surface curvatures at the discontinuities can be inferred. Simulation results on the computation of these descriptors from curved surfaces are shown. Potential applications of this representation scheme in computer vision tasks are discussed.

This paper presents efficient algorithms for updating moving octrees with real-time performance. The first algorithm works for octrees undergoing both translation and rotation motion; it works efficiently by compacting source octrees into a smaller set of cubes (not necessarily standard octree cubes) as a precomputation step, and by using a fast, exact cube/cube intersection test between source octree cubas and target octree cubes. A parallel version of the algorithm is also described. Finally, the paper presents an efficient algorithm for the more limited case of octree translation only. Experimental results are given to show the efficiency of the algorithms in comparison to competing algorithms. In addition to being fast, the algorithms presented are also space efficient in that they can produce target octrees in the linear octree representation.

Here we investigate the unique advantages of our proposed Virtual Space Teleconferencing System (VST) in the area of multimedia teleconferencing, with emphasis to facial emotion transmission and recognition. Specially, we show that this concept can be used in a unique way of communication in which the emotions of the local participant are transmitted to the remote party with higher recognition rate by enhancing the emotions using some intelligence processing in between the local and the remote participants. In other words, we can show that this kind of emotion enhanced teleconferencing systems can supersede face to face meetings, by effectively alleviating the barriers in recognizing emotions between different nations. Also in this paper we show that it is better alternative to the blurred or mosaiced facial images that one can find in some television interviews with people who are not willing to be exposed in public.

In order to display complicated virtual spaces in real time, such as spaces consisting of a dynamic natural scenery, we earlier proposed a method for simplifying the shape data of 3-D trees whereby the amount of shape data is efficiently reduced. The method generates tree shapes based on a fractal model according to the required level of details (LOD). By using a texture-mapping technique, we experimentally showed that our method can display 3-D tree images with allowable image quality in real time. However, methods for controlling the LOD of 3-D tree shapes in virtual spaces have yet to be discussed. In this paper, quantitative evaluations were made on the effect of a data simplification method employing such visual properties as resolution difference between the central vision and peripheral vision. Results showed that it is possible to display a complicated scene containing many trees in real time by controlling the LOD of tree shapes in the virtual space considering such visual properties. Furthermore, so that reality can be added to the virtual space, we consider that it is important to display the natural sways of wind-blown trees and plants in real time. Therefore, we propose a method for generating sway data for simplified tree shape data based on a simple physical model, in which each branch is connected to several other branches by springs, and also a new texture-mapping technique for rendering simplified tree shapes, making it appear as if the shapes have a high LOD. Finally, we show some examples of images of trees generated in real time by using our method, in which many trees exist and sway due to wind.

The goal of this research, as an integral component of virtual space teleconferencing systems, is to generate three-dimensional facial models from facial images and a to synthesize images of the models virtually viewed from different angles. Since there is a great gap between the images and 3 D model, we argue that it is necessary to have a base face model to provide a framework. The base model is built by carefully selecting and measuring a set of points on the face whose corresponding points can be readily identified in the input images, and another set of points that can be determined from the first point set. The input images are a front view and a side view of the face. First the extremal boundaries are extracted or interpolated, and the face features such as eyes, nose and mouth are extracted. The extracted features are then matched between the two images, and their 3 D positions calculated. Using these 3 D data, the prepared base face model is modified to approximate the face. Finally, images of the modified 3 D model are synthesized by assuming new virtual viewing angles. The originality and significance of this work lies in that the face model can be automatically generated.