Photo animation is to transform photos of real-world scenes into anime style images, which is a challenging task in AIGC (AI Generated Content). Although previous methods have achieved promising results, they often introduce noticeable artifacts or distortions. In this paper, we propose a novel double-tail generative adversarial network (DTGAN) for fast photo animation. DTGAN is the third version of the AnimeGAN series. Therefore, DTGAN is also called AnimeGANv3. The generator of DTGAN has two output tails, a support tail for outputting coarse-grained anime style images and a main tail for refining coarse-grained anime style images. In DTGAN, we propose a novel learnable normalization technique, termed as linearly adaptive denormalization (LADE), to prevent artifacts in the generated images. In order to improve the visual quality of the generated anime style images, two novel loss functions suitable for photo animation are proposed: 1) the region smoothing loss function, which is used to weaken the texture details of the generated images to achieve anime effects with abstract details; 2) the fine-grained revision loss function, which is used to eliminate artifacts and noise in the generated anime style image while preserving clear edges. Furthermore, the generator of DTGAN is a lightweight generator framework with only 1.02 million parameters in the inference phase. The proposed DTGAN can be easily end-to-end trained with unpaired training data. Extensive experiments have been conducted to qualitatively and quantitatively demonstrate that our method can produce high-quality anime style images from real-world photos and perform better than the state-of-the-art models.

Animating 3D characters using motion capture data requires basic expertise and manual labor. To support the creativity of animation design and make it easier for common users, we present a sketch-based interface DualMotion, with rough sketches as input for designing daily-life animations of characters, such as walking and jumping. Our approach enables to combine global motions of lower limbs and the local motion of the upper limbs in a database by utilizing a two-stage design strategy. Users are allowed to design a motion by starting with drawing a rough trajectory of a body/lower limb movement in the global design stage. The upper limb motions are then designed by drawing several more relative motion trajectories in the local design stage. We conduct a user study and verify the effectiveness and convenience of the proposed system in creative activities.

This paper presents an interactive locomotion controller using motion capture data and an inverted pendulum model (IPM). The motion data of a character is decomposed into those of upper and lower bodies, which are then dimension-reduced via what we call hierarchical Gaussian process dynamical model (H-GPDM). The locomotion controller receives the desired walking direction from the user. It is integrated into the IPM to determine the pose of the center of mass and the stance-foot position of the character. They are input to the H-GPDM, which interpolates the low-dimensional data to synthesise a redirected motion sequence on an uneven surface. The locomotion controller allows the upper and lower bodies to be independently controlled and helps us generate natural locomotion. It can be used in various real-time applications such as games.

The development of new technologies has undoubtedly promoted the advances of modern education, among which Virtual Reality (VR) technologies have made the education more visually accessible for students. However, classroom education has been the focus of VR applications whereas not much research has been done in promoting sports education using VR technologies. In this paper, an immersive VR system is designed and implemented to create a more intuitive and visual way of teaching tennis. A scalable system architecture is proposed in addition to the hardware setup layout, which can be used for various immersive interactive applications such as architecture walkthroughs, military training simulations, other sports game simulations, interactive theaters, and telepresent exhibitions. Realistic interaction experience is achieved through accurate and robust hybrid tracking technology, while the virtual human opponent is animated in real time using shader-based skin deformation. Potential future extensions are also discussed to improve the teaching/learning experience.

This paper introduces a new dynamic 3D mesh representation that provides 3D animation support of progressive display and drastically reduces the amount of storage space required for 3D animation. The primary purpose of progressive display is to allow viewers to get animation as quickly as possible, rather than having to wait until all data has been downloaded. In other words, this method allows for the simultaneous transmission and playing of 3D animation. Experiments show that coarser 3D animation could be reconstructed with as little as 150 KB of data transferred. Using the sustained transmission of refined operators, viewers feel that resolution approaches that of the original animation. The methods used in this study are based on a compression technique commonly used in 3D animation - clustered principle component analysis, using the linearly independent rules of principle components, so that animation can be stored using smaller amounts of data. This method can be coupled with streaming technology to reconstruct animation through iterative updating. Each principle component is a portion of the streaming data to be stored and transmitted after compression, as well as a refined operator during the animation update process. This paper considers errors and rate-distortion optimization, and introduces weighted progressive transmitting (WPT), using refined sequences from optimized principle components, so that each refinement yields an increase in quality. In other words, with identical data size, this method allows each principle component to reduce allowable error and provide the highest quality 3D animation.

This paper introduces an interactive expression editing system that allows users to design facial expressions easily. Currently, popular example-based methods construct face models based on the examples of target face. The shortcoming of these methods is that they cannot create expressions for novel faces: target faces not previously recorded in the database. We propose a solution to overcome this limitation. We present an interactive facial-geometric-feature animation system for generating expressions of novel faces. Our system is easy to use. By click-dragging control points on the target face, on the computer screen display, unique expressions are generated automatically. To guarantee natural animation results, our animation model employs prior knowledge based on various individuals' expressions. One model prior is learned from motion vector fields to guarantee effective facial motions. Another, different, model prior is learned from facial shape space to ensure the result has a real facial shape. Interactive animation problem is formulated in a maximum a posterior (MAP) framework to search for optimal results by combining the priors with user-defined constraints. We give an extension of the Motion Propagation (MP) algorithm to infer facial motions for novel target faces from a subset of the control points. Experimental results on different facial animations demonstrate the effectiveness of the proposed method. Moreover, one application of our system is exhibited in this paper, where users create expressions for facial sketches interactively.

We propose an interactive method for assisting the coloring process of 2D hand-drawn animated cartoons. It segments input frames (each hand-drawn drawing of the cartoon) into regions (areas surrounded by closed lines. E.g. the head, the hands) extracts their features, and then matches the regions between frames, allowing the user to fix coloring mistakes interactively. Its main contribution consists in storing matched regions in lists called "chains" for tracking how the region features vary along the animation. Consequently, the matching rate is improved and the matching mistakes are reduced, thus reducing the total effort needed until having a correctly colored cartoon.

"Dive into the Movie (DIM)" is a name of project to aim to realize a world innovative entertainment system which can provide an immersion experience into the story by giving a chance to audience to share an impression with his family or friends by watching a movie in which all audience can participate in the story as movie casts. To realize this system, several techniques to model and capture the personal characteristics instantly in face, body, gesture, hair and voice by combining computer graphics, computer vision and speech signal processing technique. Anyway, all of the modeling, casting, character synthesis, rendering and compositing processes have to be performed on real-time without any operator. In this paper, first a novel entertainment system, Future Cast System (FCS), is introduced which can create DIM movie with audience's participation by replacing the original roles' face in a pre-created CG movie with audiences' own highly realistic 3D CG faces. Then the effects of DIM movie on audience experience are evaluated subjectively. The result suggests that most of the participants are seeking for higher realism, impression and satisfaction by replacing not only face part but also body, hair and voice. The first experimental trial demonstration of FCS was performed at the Mitsui-Toshiba pavilion of the 2005 World Exposition in Aichi Japan. Then, 1,640,000 people have experienced this event during 6 months of exhibition and FCS became one of the most popular events at Expo.2005.

Our research goal is to facilitate the sharing of stories with digital photographs. Some map websites now collect stories associated with peoples' relationships to places. Users map collections of places and include their intangible emotional associations with each location along with photographs, videos, etc. Though this framework of mapping stories is important, it is not sufficiently expressive to communicate stories in a narrative fashion. For example, when the number of the mapped collections of places is particularly large, it is neither easy for viewers to interpret the map nor is it easy for the creator to express a story as a series of events in the real world. This is because each narrative, in the form of a sequence of textual narratives, a sequence of photographs, a movie, or audio is mapped to just one point. As a result, it is up to the viewer to decide which points on the map must be read, and in what order. The conventional framework is fairly suitable for mapping and expressing fragments or snapshots of a whole story and not for conveying the whole story as a narrative using the entire map as the setting. We therefore propose a new framework, Spatial Slideshow, for mapping personal photo collections and representing them as stories such as route guidances, sightseeing guidances, historical topics, fieldwork records, personal diaries, and so on. It is a fusion of personal photo mapping and photo storytelling. Each story is conveyed through a sequence of mapped photographs, presented as a synchronized animation of a map and an enhanced photo slideshow. The main technical novelty of this paper is a method for creating three-dimensional animations of photographs that induce the visual effect of motion from photo to photo. We believe that the proposed framework may have considerable significance in facilitating the grassroots development of spatial content driven by visual communication concerning real-world locations or events.

This letter proposes a prototyping tool for Web-based Multiuser Online Role-Playing Game (MORPG). The design goal is to make this tool simple and powerful. The tool is comprised of a GUI editor, a translator and a runtime environment. The GUI editor is used to edit state-transition diagrams, each of which defines the behavior of the fictional characters. The state-transition diagrams are translated into C program codes, which plays the role of a game engine in RPG system. The runtime environment includes PHP, JavaScript with Ajax and HTML. So the prototype system can be played on the usual Web browser, such as Firefox, Safari and IE. On a click or key press by a player, the Web browser sends it to the Web server to reflect its consequence on the screens which other players are looking at. Prospected users of this tool include programming novices and schoolchildren. The knowledge or skill of any specific programming languages is not required to create state-transition diagrams. Its structure is not only suitable for the definition of a character behavior but also intuitive to help novices understand. Therefore, the users can easily create Web-based MORPG system with the tool.

Because motion capture system enabled us to capture a number of human motions, the demand for a method to easily browse the captured motion database has been increasing. In this paper, we propose a method to generate simple visual outlines of motion clips, for the purpose of efficient motion data browsing. Our method unfolds a motion clip into a 2D stripe of keyframes along a timeline that is based on semantic keyframe extraction and the best view point selection for each keyframes. With our visualization, timing and order of actions in the motions are clearly visible and the contents of multiple motions are easily comparable. In addition, because our method is applicable for a wide variety of motions, it can generate outlines for a large amount of motions fully automatically.

This paper presents 3D keyframe animation watermarking using orientation interpolators. 3D keyframe animation consists of a number of transform nodes, including a geometrical node from the initial model and several interpolator nodes that represent object movement. Therefore, the proposed algorithm randomly selects transform nodes with orientation interpolator nodes, then resamples the quaternion components to maintain a uniform key time. Thereafter, watermark bits are embedded into quaternion components with large rotation angles. Experimental results verify the robustness of the proposed algorithm to geometrical and timeline attacks, along with the subjective and objective quality of its invisibility.

This paper investigates the use of the affine transformation matrix when employing principal component analysis (PCA) to compress the data of 3D animation models. Satisfactory results were achieved for the common 3D models by using PCA because it can simplify several related variables to a few independent main factors, in addition to making the animation identical to the original by using linear combinations. The selection of the principal component factor (also known as the base) is still a subject for further research. Selecting a large number of bases could improve the precision of the animation and reduce distortion for a large data volume. Hence, a formula is required for base selection. This study develops an automatic PCA selection method, which includes the selection of suitable bases and a PCA separately on the three axes to select the number of suitable bases for each axis. PCA is more suitable for animation models for apparent stationary movement. If the original animation model is integrated with transformation movements such as translation, rotation, and scaling (RTS), the resulting animation model will have a greater distortion in the case of the same base vector with regard to apparent stationary movement. This paper is the first to extract the model movement characteristics using the affine transformation matrix and then to compress 3D animation using PCA. The affine transformation matrix can record the changes in the geometric transformation by using 44 matrices. The transformed model can eliminate the influences of geometric transformations with the animation model normalized to a limited space. Subsequently, by using PCA, the most suitable base vector (variance) can be selected more precisely.

An accurate audio-visual speech corpus is inevitable for talking-heads research. This paper presents our audio-visual speech corpus collection and proposes a head-movement normalization method and a facial motion generation method. The audio-visual corpus contains speech data, movie data on faces, and positions and movements of facial organs. The corpus consists of Japanese phoneme-balanced sentences uttered by a female native speaker. An accurate facial capture is realized by using an optical motion-capture system. We captured high-resolution 3D data by arranging many markers on the speaker's face. In addition, we propose a method of acquiring the facial movements and removing head movements by using affine transformation for computing displacements of pure facial organs. Finally, in order to easily create facial animation from this motion data, we propose a technique assigning the captured data to the facial polygon model. Evaluation results demonstrate the effectiveness of the proposed facial motion generation method and show the relationship between the number of markers and errors.

Animation video is becoming very popular with the availability of low cost computing resources. The cell animation is a popular method, used for producing animation video. In order to efficiently encode these videos, conventional video codecs, such as AutoDesk Animation Pro FLC, Intel Indeo 5, and MPEG-4 can be used to achieve high compression. However, when cell animation videos are compressed at very low bit rate by these traditional codecs, objectionable artifacts, e.g., false color, blurred contours, and blocking artifact, are severely occurred. In this paper, we propose an efficient compression method for cell animation images. The proposed method employs hybrid coding scheme which includes intraframe and interframe coding modes. The intraframe mode consists of color quantization, adaptive differential pulse code modulation, forward classification, and Golomb-Rice coding. The interframe coding consists of block-based techniques and exploits the characteristics of motion. Simulation results show that the proposed method provides superior performance over AutoDesk Animation Pro FLC, MPEG-1, Intel Indeo 5, and MPEG-4 standards.

One of the most challenging tasks in computer graphics and cyberworlds is the realistic animation of the behavior of virtual agents emulating human beings and evolving within virtual 3D worlds. In a previous paper, the authors presented a new, sophisticated behavioral system that allows the agents to take intelligent decisions by themselves. A central issue of this process is the adequate choice of appropriate mechanisms for goal selection. This is actually the aim of the present contribution. In this paper a new scheme for goal selection is described. According to it, the goal's priority is calculated as a combination of different agent's internal states (given by mathematical functions also described in this paper) and external factors (which will determine the goal's feasibility). The architecture of the goal selection module as well as its simulation flow are also analyzed in this paper. Finally, the excellent performance of this new scheme is enlightened by means of a simple yet illustrative example.

Spring-mass systems are widely used in computer animation to model soft objects. Although the systems can be numerically solved either by explicit methods or implicit methods, it has been difficult to obtain stable results from explicit methods. This paper describes detailed discussion on stabilizing explicit methods in spring-mass simulation. The simulation procedures are modeled as a linear digital system, and system stability is mathematically defined. This allows us to develop theories of simulation stability. The application of these theories to explicit methods allows them to become as stable as implicit methods. Furthermore, a faster explicit method is proposed. Experiments confirm the theories and demonstrate the efficiency of the proposed methods.

In this study, a CG animation tool was designed that allows interpolation and extrapolation of two types of repeated motions including finger actions, for quantitative analyses of the relationship between features of human motions and subjective impressions. Three-dimensional human motions are measured with a magnetic motion capture and a pair of data gloves, and then relatively accurate time-series joint data are generated utilizing statistical characteristics. Based on the data thus obtained, time-series angular data of each joint for two dancing motions is transformed into frequency domain by Fourier transform, and spectral shape of each dancing action is interpolated. The interpolation and extrapolation of two motions can be synthesized with simple manner by changing an weight parameter while keeping good harmony of actions. Using this CG animation tool as a motion synthesizer, repeated human motions such as a dancing action that gives particular impressions on the observers can be quantitatively measured and analyzed by the synthesis of actions.

In this paper, we propose a new technique to generate virtual views of three-dimensional (3D) models. The technique is implemented into our facial pose transformation system, which takes only one frontal image and transforms it into virtual views. In our system, to overcome the complex of 3D geometric model, Image Based Rendering based algorithm and mesh-based methods are applied. We also introduce our new Invertible Meshwarp Algorithm, which is developed based on Two-pass Meshwarp Algorithm. Firstly, in the system, for any given person, we take a frontal face image to compose a frontal mesh for it. The standard mesh set of a specific person is created for several face sides; front, half left, half right, left and right side. The other meshes are then automatically generated based on the standard mesh set and the frontal mesh. Continually, we use Invertible Meshwarp Algorithm, which improvably solves the overlap or inversion of neighbor vertices of those created meshes. This step will finalize the generation of different views or the virtual looks of the frontal face image. We then evaluate our transformation system performance by comparing the normalized distance between several feature points in the real and transformed face images. The system is built based on C/C++ language and our result shows that the average error in the feature location is about 7% of the distance from the center of both eyes to the center of a mouth between the actual and transformed face images.

Although watermarking techniques have been extensively developed for natural videos, little progress is made in the area of graphics animation. Following the former successful MPEG-1 and MPEG-2 coding standards that provide efficient representations of natural videos, the emerging MPEG-4 standard incorporates new coding tools for 2D mesh animation. Graphics animation information is crucial for many applications and may need proper protection. In this paper, we develop a watermarking technique suitable for MPEG-4 2D mesh animation. The proposed method is based on the multiresolution analysis of 2D dynamic mesh. We perform wavelet transform on the temporal sequence of the node points to extract the significant spectral components of mesh movement, which we term the "feature motions. " A binary watermark invisibly resides in the feature motions based on the spread-spectrum principle. Before watermark detection, a spatial-domain least-squares registration technique is used to restore the possibly geometrically distorted mesh data. Each watermark bit is then detected by hard decision with cryptographically secure keys. We have tested the proposed method with a variety of attacks, including affine transformations, temporal smoothing, spectral enhancement and attenuation, additive random noise, and a combination of the above. Experimental results show that the proposed watermarks can withstand the aforementioned attacks.