A method to synthesize facial caricatures with non-planar expression is proposed. Several methods have been already proposed to synthesize facial caricatures automatically, but they mainly synthesize plane facial caricatures which look somewhat monotonous. In order to generate expressive facial caricature, the image should be expressed in non-planar style, expressing the depth of the face by shading and highlighting. In this paper, a new method to express such non-planar effect in facial caricatures is proposed by blending the grayscale information of the real face image into the plane caricature. Some methods also have been proposed to generate non-planar facial caricature, but the proposed method can adjust the degree of non-planar expression by interactive evolutionary computing, so that the obtained expression is satisfied by the user based on his/her subjective criteria. Since the color of the face looks changed, when the grayscale information of the natural face image is mixed, the color information of the skin area are also set by interactive evolutionary computing. Experimental results show the high performance of the proposed method.

This paper surveys the results of various studies on 3-D image coding. Themes are focused on efficient compression and display-independent representation of 3-D images. Most of the works on 3-D image coding have been concentrated on the compression methods tuned for each of the 3-D image formats (stereo pairs, multi-view images, volumetric images, holograms and so on). For the compression of stereo images, several techniques concerned with the concept of disparity compensation have been developed. For the compression of multi-view images, the concepts of disparity compensation and epipolar plane image (EPI) are the efficient ways of exploiting redundancies between multiple views. These techniques, however, heavily depend on the limited camera configurations. In order to consider many other multi-view configurations and other types of 3-D images comprehensively, more general platform for the 3-D image representation is introduced, aiming to outgrow the framework of 3-D "image" communication and to open up a novel field of technology, which should be called the "spatial" communication. Especially, the light ray based method has a wide range of application, including efficient transmission of the physical world, as well as integration of the virtual and physical worlds.

To make the model-based coding a practical method, new signal processing techniques other than fully-automatic image recognition should be studied. Also after having realized the model-based coding, another new signal processing technique to improve the performance of the model-based coding should be studied. Moreover non-coding functions related to the model-based coding can be embedded as additional features. The authors are studying the interactive model-based coding in order to achieve its practical realization, improve its performance and extend related non-coding functions. We have already proposed the basic concept of interactive model-based coding and presented an eyeglasses processing for a facial image with glasses to remove the frame for improving the model-based coding performance. In this paper, we focus on the 3-D motion detection algorithm in the interactive model-based coding. Previous works were mainly based on iterative methods to solve non-linear equations. A new motion detection algorithm is developed for interactive model-based coding. It is linear because the interactive operation generates more information and the environment of the applications limits the range of parameters. The depth parameter is first obtained by the fact that a line segment is invariant as to 3-D space transformation. Relation of distance between two points is utilized. The number of conditions is larger than that of the unknown variables, which allows to use least square method for obtaining stable solutions in the environment of the applications. Experiments are carried out using the proposed motion detection method and input noise problems are removed. Synthesized wireframe modified by eight parameters provides smooth and natural motion.

This paper is concerned with the data compression and interpolation of multi-view image set. In this paper, we propose a novel disparity compensation scheme based on geometric relationship. We first investigate the geometric relationship between a point in the object space and its projection onto view images. Then, we propose the disparity compensation scheme which utilize the geometric constraints between view images. This scheme is used to compress the multi-view image into the structure of the triangular patches and the texture data on the surface of patches. This scheme not only compresses the multi-view image but also synthesize the view images from any viewpoints in the viewing zone. Also, this scheme is fast and have compatibility with 2-D interframe coding. Finally, we report the experiment, where two sets multi-view image were used as original images and the amount of data was reduced to 1/19 and 1/20 with SNR 34 dB and 20 dB, respectively.

We describe an approach to describe moving pictures in terms of their structural properties for video editing, video indexing, and video coding. The description contains 2D shape, motion, spatial relation, and relative depth of each region. To obtain the description, we develop the incremental segmentation scheme which includes dynamic occlusion analysis to determine relative depths of several objects. The scheme has been designed along the analysis-by-synthesis" approach, and uses a sequence of images to estimate object boundaries and motion information successively/incrementally. The scheme consists of three components: motion estimation, prediction with dynamic occlusion analysis, and update of the segmentation results. By combining the information from extended (longer) image sequences, and also by treating the segmentation and dynamic occlusion analysis simultaneously, the scheme attempts to improve successively over time the accuracy of the object boundary and motion estimation.

We have recently extended one of the conceptions of the lossless universal pattern-matching coding, viz. the concept of coding via copying, to multi-dimensional lossy coding, and applied the extended concept to intraframe compression of still images. The work herein applies the extended concept of lossy coding via copying to interframe low-rate video compression, thus developing a novel low-rate interframe PMIC (pattern-matching image coding) technique, which produces the effect of generalizing the definition of a search area used in the existing block-matching motion compensation. We have experimentally shown the performance gain provided by the generalization within the framework of lossy coding via copying, and demonstrated that the interframe PMIC technique is usefull and potential as a basic means for low-rate video compression.

Despite its potential to realize image communication at extremely low rates, model-based coding (analysis-synthesis coding) still has problems to be solved for any practical use. The main problems are the difficulty in modeling unknown objects and the presence of analysis errors. To cope with these difficulties, we incorporate waveform coding into model-based coding (model-based/waveform hybrid coding). The incorporated waveform coder can code unmodeled objects and cancel the artifacts caused by the analysis errors. From a different point of view, the performance of the practically used waveform coder can be improved by the incorporation of model-based coding. Since the model-based coder codes the modeled part of the image at extremely low rates, more bits can be allocated for the coding of the unmodeled region. In this paper, we present the basic concept of model-based/waveform hybrid coding. We develop a model-based/MC-DCT hybrid coding system designed to improve the performance of the practically used MC-DCT coder. Simulation results of the system show that this coding method is effective at very low transmission rates such as 16kb/s. Image transmission at such low rates is quite difficult for an MC-DCT coder without the contribution of the model-based coder.

We describe an approach for modelling a person's face for model-based coding. The goal is to estimate the 3D shape by combining the contour analysis and shading analysis of the human face image in order to increase the quality of the estimated 3D shape. The motivation for combining contour and shading cues comes from the observation that the shading cue leads to severe errors near the occluding boundary, while the occluding contour cue provides incomplete surface information in regions away from contours. Towards this, we use the deformable model as the common level of integration such that a higher-quality measurement will dominate the depth estimate. The feasibility of our approach is demonstrated using a real facial image.

Recently, cerebral disease is being a serious problem in an aging society. But, rank evaluation of cerebral disease is not developed and therefore rehabilitation is hard. In this study, we try to assess slight cerebral disease by taking notice of recognition mechanism of face and realizing face image synthesis using computer technology. If we can find a slight cerebral disease and rank evaluation, we can apply to rehabilitation, and a load of medical doctor and patient decreases. We have obtained a result by the experiment, so we report it.

Recently, studies aiming at the next generation of visual communication services which support better human communication have been carried out intensively in Japan. The principal motive of these studies is to develop new services which are not restricted to a conventional communication framework based on the transmission of waveform signals. This paper focuses on three important key words in these studies; "intelligent," "real," and "distributed and collaborative," and describes recent research activities. The first key word "intelligent" relates to intelligent image coding. As a particular example, model-based coding of moving facial images is discussed in detail. In this method, shape change and motion of the human face is described by a small number of parameters. This feature leads to the development of new applications such as very low bit-rate transmission of moving facial images, analysis and synthesis of facial expression, human interfaces, and so on. The second key word "real" relates to communication with realistic sensations and virtual space teleconferencing. Among various component technologies, real-time reproduction of 3-D human images and a cooperative work environment with virtual space are discussed in detail. The last key word "distributed and collaborative" relates to collaborative work in a distributed work environment. The importance of visual media in collaborative work, a concept of CSCW, and requirements for realizing a distributed collaborative environment are discussed. Then, four examples of CSCW systems are briefly outlined.

A function can be represented in many ways. A representation O of a function f is called 'obscure' if O is different from the representation D used as the definition of f and if it is (or, seems to be) computationally infeasible to get D from O. Such an obscure representation is useful for cryptographic techniques so that it is important to estimate its descriptive and executive complexity. We present a complexity-estimation method for certain functions used to constructing asymmetric cryptosystems. Let m be a positive integer and let K, Km, and L denote the field {0, 1}, the set of all m-tuples over K, and an extention field or order m over K, respectively. The objective function is a composit g:Km Km of three functions s, e, and t, where s:Km L and t:L Km are affine and e:L L is defined by a univariate polynomial e over L. The obscure representation of g is an m-tuple g of m-variate polynomials over K. The complexity respect to g is well measured by its degree. So we give a theorem for estimating the degree of g in terms of a characteristic quantity of the polynomial e.

This paper deals with the recent trends of reseaches on intelligent image coding technology focusing on model-based analysis synthesis coding. By means of the intelligent image coding scheme, we will be able to realize epock-making ultra-low-rate image transmission and/or so-called value-added visual telecommunications. In order to categorize the various image coding systems and examine their potential applications in the future, an approach to define generations of image coding technologies is presented. The future generation coding systems include the model-based analysis synthesis coding and knowledge-based intelligent coding. The latter half of the paper will be devoted to the recent work of the authors on the model-based analysis-synthesis coding system for facial images.

This paper discusses recognition up to intensities of mix of primary facial expressions in real time. The proposed recognition method is compatible with the MPEG-4 high level expression Facial Animation Parameter (FAP). In our method, the whole facial image is considered as a single pattern without any block segmentation. As model features, an expression vector, viz. low global frequency coefficient (DCT) changes relative to neutral facial image of a person is used. These features are robust and good enough to deal with real time processing. To construct a person specific model, apex images of primary facial expression categories are utilized as references. Personal facial expression space (PFES) is constructed by using multidimensional scaling. PFES with its generalization capability maps an unknown input image relative to known reference images. As PFES possesses linear mapping characteristics, MPEG-4 high level expression FAP can be easily calculated by the location of the input face on PFES. Also, temporal variations of facial expressions can be seen on PFES as trajectories. Experimental results are shown to demonstrate the effectiveness of the proposed method.

This paper addresses thecurrent research and recent trends of image communications research that fall into the theme of intelligent communications in two parts --Intelligent Image Coding and Communications with Realistic Sensations. In Part , the theme of intelligent image coding and the particular example of encoding of human facial images using an intelligence-based approach is presented. A classification of the different generations--five generations--of image coding systems is first given. First and second generations are the direct wave-form coding and the statistical encoding methods. The third and fourth generations are the use of model-based anaylsis-synthesis and recognition/reconstruction methods, respectively. The fifth generation is the full intelligent coding system that can understand the semantics, intention, motives and other factors in an image. The progress in the model-based analysis-synthesis approach to image coding is reviewed and the particular example of its application to the encoding of human facial images is examined. In this example, the methods of modeling human face, the extraction of feature points and the tracing of motion in a facial image, as well as the analysis and synthesis of facial expressions are discussed. In Part , the current research in the new paradigm of telecommunication--Communications with Realistic Sensations is introduced. The progress in the research of a Virtual Space Teleconference System as an example of this new paradigm is presented. The possibility of delivering a very high level of sensation realism to a receiver is made real by recent advances in computer graphics and computer animation techniques, in stereoscopic display techniques that enable life-size images to be displayed on a wide screen and in computer human interface technologies, particularly, for recording human hand, head and eye positions. Progress of these technologies in the context of the research on the Virtual Space Teleconference System are discussed. The research on how to communicate and manipulate 3-D objects in a such an environment is examined.