Action recognition using skeleton data (3D coordinates of human joints) is an attractive topic due to its robustness to the actor's appearance, camera's viewpoint, illumination, and other environmental conditions. However, skeleton data must be measured by a depth sensor or extracted from video data using an estimation algorithm, and doing so risks extraction errors and noise. In this work, for robust skeleton-based action recognition, we propose a deep state-space model (DSSM). The DSSM is a deep generative model of the underlying dynamics of an observable sequence. We applied the proposed DSSM to skeleton data, and the results demonstrate that it improves the classification performance of a baseline method. Moreover, we confirm that feature extraction with the proposed DSSM renders subsequent classifications robust to noise and missing values. In such experimental settings, the proposed DSSM outperforms a state-of-the-art method.

This paper proposes a method for predicting individuality-preserving gait patterns. Physical rehabilitation can be performed using visual and/or physical instructions by physiotherapists or exoskeletal robots. However, a template-based rehabilitation may produce discomfort and pain in a patient because of deviations from the natural gait of each patient. Our work addresses this problem by predicting an individuality-preserving gait pattern for each patient. In this prediction, the transition of the gait patterns is modeled by associating the sequence of a 3D skeleton in gait with its continuous-value gait features (e.g., walking speed or step width). In the space of the prediction model, the arrangement of the gait patterns are optimized so that (1) similar gait patterns are close to each other and (2) the gait feature changes smoothly between neighboring gait patterns. This model allows to predict individuality-preserving gait patterns of each patient even if his/her various gait patterns are not available for prediction. The effectiveness of the proposed method is demonstrated quantitatively. with two datasets.

This paper proposes a method for evaluating a physical gait motion based on a 3D human skeleton measured by a depth sensor. While similar methods measure and evaluate the motion of only a part of interest (e.g., knee), the proposed method comprehensively evaluates the motion of the full body. The gait motions with a variety of physical disabilities due to lesioned body parts are recorded and modeled in advance for gait anomaly detection. This detection is achieved by finding lesioned parts a set of pose features extracted from gait sequences. In experiments, the proposed features extracted from the full body allowed us to identify where a subject was injured with 83.1% accuracy by using the model optimized for the individual. The superiority of the full-body features was validated in in contrast to local features extracted from only a body part of interest (77.1% by lower-body features and 65% by upper-body features). Furthermore, the effectiveness of the proposed full-body features was also validated with single universal model used for all subjects; 55.2%, 44.7%, and 35.5% by the full-body, lower-body, and upper-body features, respectively.

A novel method is proposed to estimate the 3D relative positions of an articulated body from point correspondences in an uncalibrated monocular image sequence. It is based on a camera perspective model. Unlike previous approaches, our proposed method does not require camera parameters or a manual specification of the 3D pose at the first frame, nor does it require the assumption that at least one predefined segment in every frame is parallel to the image plane. Our work assumes a simpler assumption, for example, the actor stands vertically parallel to the image plane and not all of his/her joints lie on a plane parallel to the image plane in the first frame. Input into our algorithm consists of a topological skeleton model and 2D position data on the joints of a human actor. By geometric constraint of body parts in the skeleton model, 3D relative coordinates of the model are obtained. This reconstruction from 2D to 3D is an ill-posed problem due to non-uniqueness of solutions. Therefore, we introduced a technique based on the concept of multiple hypothesis tracking (MHT) with a motion-smoothness function between consecutive frames to automatically find the optimal solution for this ill-posed problem. Since reconstruction configurations are obtained from our closed-form equation, our technique is very efficient. Very accurate results were attained for both synthesized and real-world image sequences. We also compared our technique with both scaled-orthographic and existing perspective approaches. Our proposed method outperformed other approaches, especially in scenes with strong perspective effects and difficult poses.

In recent years, digital maps have been used in a variety of scenarios, including car navigation systems and map information services over the Internet. These digital maps are formed by multiple layers of maps of different scales; the map data most suitable for the specific situation are used. Currently, the production of map data of different scales is done by hand due to constraints related to processing time and accuracy. We conducted research concerning technologies for automatic generation of simplified map data from detailed map data. In the present paper, the authors propose the following: (1) a method to transform data related to streets, rivers, etc. containing widths into line data, (2) a method to eliminate the component points of the data, and (3) a method to eliminate data that lie below a certain threshold. In addition, in order to evaluate the proposed method, a user survey was conducted; in this survey we compared maps generated using the proposed method with the commercially available maps. From the viewpoint of the amount of data reduction and processing time, and on the basis of the results of the survey, we confirmed the effectiveness of the automatic generation of simplified maps using the proposed methods.

A novel pedestrian tracking scheme based on a particle filter is proposed, which adopts a skeleton model of a pedestrian for a state space model and distance transformed images for likelihood computation. The 6-stick skeleton model used in the proposed approach is very distinctive in representing a pedestrian simply but effectively. By the experiment using the real sequences provided by PETS, it is shown that the target pedestrian is tracked adequately by the proposed approach with a simple silhouette extraction method which consists of only background subtraction, even if the tracking target moves so complicatedly and is often so cluttered by other obstacles that the pedestrian can not be tracked by the conventional methods. Moreover, it is demonstrated that the proposed scheme can track the multiple targets in the complex case that their trajectories intersect.

Pneumatic pressure, which is easy enough to be handled in comparison with hydraulic pressure and is endowed with high safety, is available for a power source of a robot arm to be utilized in concert with human beings to do various types of work. But pneumatic pressure is so low in comparison with hydraulic pressure that an air cylinder having a diameter long enough and stroke wide enough is required to obtain great output power. In this study, therefore, the investigation was made with layout of air cylinders and transmission mechanisms of the motion power directed toward the driving joints to be followed by development of a new humanoid robot arm with seven degrees of freedom in which air cylinders are compactly incorporated. To be concrete with this, contrivance was made with an endoskeleton structure allowing almost all of the structure materials of the individual arm joints to be shared by the air cylinder with incorporation of the air cylinder in the axes of the upper arm joint and forearm joints by paying attention to the fact that the cylinder itself has high strength. The evaluation experiments driving the robot arm referred to above were conducted by means of I-PD control. The results suggested that the mechanism of the robot with seven degrees of freedom having pneumatic actuators proposed in this study is useful as the humanoid robot arm. The quick and accurate motions were accomplished with I-PD control which is relatively easy to be dealt with but not suitable for non-linear actuator system.

Developing embedded parallel image processing applications is usually a very hardware-dependent process, often using the single instruction multiple data (SIMD) paradigm, and requiring deep knowledge of the processors used. Furthermore, the application is tailored to a specific hardware platform, and if the chosen hardware does not meet the requirements, it must be rewritten for a new platform. We have proposed the use of design space exploration [9] to find the most suitable hardware platform for a certain application. This requires a hardware-independent program, and we use algorithmic skeletons [5] to achieve this, while exploiting the data parallelism inherent to low-level image processing. However, since different operations run best on different kinds of processors, we need to exploit task parallelism as well. This paper describes how we exploit task parallelism using an asynchronous remote procedure call (RPC) system, optimized for low-memory and sparsely connected systems such as smart cameras. It uses a futures [16]-like model to present a normal imperative C-interface to the user in which the skeleton calls are implicitly parallelized and pipelined. Simulation provides the task dependency graph and performance numbers for the mapping, which can be done at run time to facilitate data dependent branching. The result is an easy to program, platform independent framework which shields the user from the parallel implementation and mapping of his application, while efficiently utilizing on-chip memory and interconnect bandwidth.

In this paper, a process is described for analysing the motion of a human target in a video stream. Moving targets are detected and their boundaries extracted. From these, a "star" skeleton is produced. Two motion cues are determined from this skeletonization: body posture, and cyclic motion of skeleton segments. These cues are used to determine human activities such as walking or running, and even potentially, the target's gait. Unlike other methods, this does not require an a priori human model, or a large number of "pixels on target". Furthermore, it is computationally inexpensive, and thus ideal for real-world video applications such as outdoor video surveillance.

The skeleton and the skeleton function of an object are important representations for shape analysis and recognition. They contain enough information to recognize an object and to reconstruct its original shape. However, they are sensitive to distortion caused by rotation and noise. This paper presents another approach for binary object representation called a modified exoskeleton(mES) that combines the previously defined exoskeleton with the use of symmetric object whose dominant property is rotation invariant. The mES is the skeleton of a circular background around the object that preserves the skeleton properties including significant information about the object for use in object recognition. Then the matching algorithm for object recognition based on the mES is presented. We applied the matching algorithm to evaluate the mES against the skeleton obtained from using 4-neighbor distance transformation on a set of artificial objects, and the experimental results reveal that the mES is more robust to distortion caused by rotation and noise than the skeleton and that the matching algorithm is capable of recognizing objects effectively regardless of their size and orientation.

In this paper we propose a method for generating Prolog program code and skeleton C code from a specification of requirements written in DFDs (Data Flow Diagram) and DD (Data Dictionary). This generation of code takes two transformation steps. The specification is transformed into a Prolog program and the transformed Prolog is used for generating skeleton C code so that the specification is directly expendable in the conventional programming environment. This work makes it possible to rapidly have a prototype by executing Prolog programs and remove the design stage from the software development life cycle. This has been implemented on UNIX workstation environment with a data flow diagram editor START system.

We thought that multiple skeletons were inherent in an ordinary three-dimensional object. A thinning method is developed to extract multiple skeletons using 333 templates for boundary deletion based on the hit or miss transformation and 222 templates for checking one voxel thickness. We prepared twelve sets of deleting templates consisting of total 194 templates and 72 one voxel checking templates. One repetitive iteration using one sequential use of the template sets extracts one skeleton. Some of the skeletons thus obtained are identical; however, multiple independent skeletons are extracted by this method. These skeletons fulfill the well-recognized three conditions for a skeleton. We extracted three skeletons from the cube, two from the space shuttle model and four from the L-shaped figure by Tsao and Fu. The digital medial skeleton, which is not otherwise extracted, is extracted by comparing the multiple skeletons with the digital medial-axis-like-figure. One of our skeletons for the cude agreed with the ideal medial axis. The locations of the gravity center of the multiple skeletons are compared with that of the original shape to evaluate how uniform or non-biased skeletons are extracted. For the L-shaped figure, one of our skeletons is found to be most desirable from the medial and uniform points of view.

In this paper, we present new algorithms to calculate the reverse distance transformation and to extract the skeleton based upon the Euclidean metric for an arbitrary binary picture. The presented algorithms are applicable to an arbitrary picture in all of n-dimensional spaces (n2) and a digitized picture sampled with the different sampling interval in each coordinate axis. The reconstruction algorithm presented in this paper is resolved to serial one-dimensional operations and efficiently executed by general purpose computer. The memory requirement is very small including only one picture array and single one-dimensional work space array for n-dimensional pictures. We introduce two different definitions of skeletons, both of them allow us to reconstruct the original binary picture exactly, and present algorithms to extract those skeltons from the result of the squared Euclidean distance transformation.

This paper presents a new line extraction method to capture vectors based on contours and skeletons from line drawing raster images in which the lines are touched by characters or other lines. Conventionally, two line extraction methods have generally been used. One is a thinning method. The other is a medial line extraction method based on parallel pairs of contours. The thinning method tends to distort the extracted lines, especially at intersections and corners. On the other hand, the medial line extraction method has a poor capability as regards capturing correct lines at intersections. Contours are able to maintain edge shapes well, while skeletons preserve topological features; thus, a combination of these features effectively leads to the best fitting line. In the proposed method, the line which best fits the original image is selected from among various candidate lines. The candidates are created from several merged short skeleton fragments located between pairs of short contour fragments. The method is also extended to circular arc fitting. Experimental results show that the proposed line fitting method is effective.