At present, the generative adversarial network (GAN) plays an important role in learning tasks. The basic idea of a GAN is to train the discriminator and generator simultaneously. A GAN-based inverse tone mapping method can generate high dynamic range (HDR) images corresponding to a scene according to multiple image sequences of a scene with different exposures. However, subsequent tone mapping algorithm processing is needed to display it on a general device. This paper proposes an end-to-end multi-exposure image fusion algorithm based on a relative GAN (called RaGAN-EF), which can fuse multiple image sequences with different exposures directly to generate a high-quality image that can be displayed on a general device without further processing. The RaGAN is used to design the loss function, which can retain more details in the source images. In addition, the number of input image sequences of multi-exposure image fusion algorithms is often uncertain, which limits the application of many existing GANs. This paper proposes a convolutional layer with weights shared between channels, which can solve the problem of variable input length. Experimental results demonstrate that the proposed method performs better in terms of both objective evaluation and visual quality.

We propose an image restoration technique that uses multiple image integration. The detail of the dark area when acquiring a dark scene is often deteriorated by sensor noise. Simple image integration inherently has the capability of reducing random noises, but it is especially insufficient in scenes that have a dark area. We introduce a novel image integration technique that optimizes the weights for the integration. We find the optimal weight map by solving a convex optimization problem for the weight optimization. Additionally, we apply the proposed weight optimization scheme to a single-image super-resolution problem, where we slightly modify the weight optimization problem to estimate the high-resolution image from a single low-resolution one. We use some of our experimental results to show that the weight optimization significantly improves the denoising and super-resolution performances.

High dynamic range (HDR) images that include large differences in brightness levels are studied to address the lack of knowledge on the quality estimation method for real HDR images. For this, we earlier proposed a new metric, the independent signal-to-noise ratio (ISNR), using the independent pixel value as the signal instead of the peak value (PSNR). Next, we proposed the local peak signal-to-noise ratio (LPSNR), using the maximum value of neighboring pixels, as an improved version. However, these methods did not sufficiently consider human perception. To address this issue, here we proposed an objective estimation method that considers spatial frequency characteristics based on the actual brightness. In this method, the approximated function for human characteristics is calculated and used as a 2D filter on an FFT for spatial frequency weighting. In order to confirm the usefulness of this objective estimation method, we compared the results of the objective estimation with a subjective assessment. We used the organic EL display which has a perfect contrast ratio for the subjective assessment. The results of experiments showed that perceptual weighting improves the correlation between the SNR and MOS of the subjective assessment. It is recognized that the weighted LPSNR gives the best correlation.

In this paper we propose a new tone mapping method for HDR video. Two types of gamma tone mapping are blended to preserve local contrast in the entire range of luminance. Our method achieves high quality tone mapping especially for the HDR video that has a nonlinear response to scene radiance. Additionally, we apply it to an object-aware tone mapping method for camera surveillance. This method achieves high visibility of target objects in the tone mapped HDR video. We examine the validity of our methods through simulation and comparison with conventional work.

The purpose of this paper is to present the novel technique to solve the recognition errors in invariant range image multi-pose face recognition. The scale, center and pose error problems were solved by using the geometric transform. Range image face data (RIFD) was obtained from a laser range finder and was used in the model to generate multi-poses. Each pose data size was reduced by linear reduction. The reduced RIFD was transformed to the gradient face model for facial feature image extraction and also for matching using the Membership Matching Score model. Using this method, the results from the experiment are acceptable although the size of gradient face image data is quite small (659 elements). Three-Layer Matching Search was the algorithm designed to reduce the access timing to the most accurate and similar pose position. The proposed algorithm was tested using facial range images from 130 people with normal facial expressions and without eyeglasses. The results achieved the mean success rate of 95.67 percent of 12 degrees up/down and left/right (UDLR) and 88.35 percent of 24 degrees UDLR.

In this paper, we discuss noise reduction approaches to improving range images using a nonlinear 2D Kalman filter. First, we propose the nonlinear 2D Kalman filter, which can reduce noise in the range image using an estimated edge vector and a nonlinear function that does not distort sharp edges. Second, we evaluate reduction of the additive noise in a test range image using the mean square error (MSE). Third, we discuss the detection rate and the number of false detections in the estimated range image. Fourth, a simulation example demonstrating the performance of the proposed 2D Kalman filter for a real range image having abrupt changes is presented. Finally, simulation results are presented which show that the estimated image of the nonlinear 2D Kalman filter is effective in reducing the amount of noise, while causing minimal smoothing of the abrupt changes.

The problem of detecting straight lines arises frequently in image processing and computer vision. Here we consider the problem of extracting lines from range images and more generally from sets of three-dimensional (3D) points. The problem is stated as follows. Given a set Γ of points in 3D space and a non-negative constant , determine the line that is at a distance at most ε from the maximal number of points of . The extraction of multiple lines is achieved iteratively by performing this best line detection and removing at each iteration the points that are close to the line found. We consider two approaches to solve the problem. The first is a simple approach that selects the best line among a randomly chosen subset of lines each defined by a pair of edge points. The second approach, based on tabu search, explores a larger set of candidate lines thus obtaining a better fit of the lines to the points. We present experimental results on different types of three-dimensional data (i) range images of polyhedral objects (ii) secondary structures (helices and strands) of large molecules.

In this paper, a method of fusing ground-based laser range image and CCD images for the reconstruction of textured 3D urban object is proposed. An acquisition system is developed to capture laser range image and CCD images simultaneously from the same platform. A registration method is developed using both laser range and CCD images in a coarse-to-fine process. Laser range images are registered with an assumption on sensor's setup, which aims at robustly detecting an initial configuration between the sensor's coordinate system of two views. CCD images are matched to refine the accuracy of the initial transformation, which might be degraded by improper sensor setup, unreliable feature extraction, or limited by low spatial resolution of laser range image. Textured 3D model is generated using planar faces for vertical walls and triangular cells for ground surface, trees and bushes. Through an outdoor experiment of reconstructing a building using six views of laser range and CCD images, it is demonstrated that textured 3D model of urban objects can be generated in an automated manner.

When an image of a 3D object is transmitted or recorded, its range image as well its grey-scale image are required. In this paper, we propose a method of coding for efficient compression of a pair of a pair of range and grey-scale images of 3D objects. We use Lambertian reflection optical model to model the relationship between a 3D object shape and it's brightness. Good illuminant direction estimation leads to good grey-scale image generation and furthermore effects compression results. A method for estimating the illuminant derection and composite albedo from grey-scale image statistics is presented. We propose an approach for estimating the slant angle of illumination based on an optical model from a pair of range and grey-scale images. Estimation result shows that our approach is better. Using the estimated parameters of illuminant direction and composite albedo a synthetic grey-scale image is generated. For comparison, a comparison coding method is used, in which we assume that the range and grey-scale images are compressed separately. We propose an efficient compression coding method for a pair of range and grey-scale images in which we use the correlation between range and grey-scale images, and compress them together. We also evaluate the coding method on a workstation and show numerical results.

This paper proposes a new approach to deal with the various quality of the reference impressions by measuring the seal to register as 3D (three-dimensional) image, that is, range image. By registering a seal as 3D image, it becomes possible to construct various 2D impressions from it according to the affixing conditions of the reference impression such as the affixing slant, the affixing pressure, the state of the ink on the seal surface and so on. Then, the accurate and easy identification of the seals will be possible by comparing the constructed impression with the reference impression. The performance is verified by experiment, and the result shows that plural 2D impressions according to the affixing conditions can be constructed from only one 3D image of the registered seal.

In this paper, we present a digital scheme for fast VLSI range imaging sensor, which is a modification of the analog scheme of existing sensor implemented by T. Kanade. Instead of reading timing information in analog manner, we use a digital scheme which has several advantages over the analog scheme, including area saving, insusceptibility to noise and other undesirable effects. We have implemented a prototype to test feasibility and present its experimental result.

A method of range image segmentation using four Markov random field(MRF)s is described in this paper. MRFs are used in depth smoothing, gradient smoothing, edge detection and surface type labeling stage. First, range and its gradient images are smoothed preserving jump and roof edges respectively using line process concept one after another. Then jump and roof edges are extracted, combined and refined using penalizing undesirable edge patterns. Finally, curvatures are computed and the surface types are labeled according to the signs of principal curvatures. The surface type labels are refined using winner-takes-all layers in the stage. The final output is a set of regions with its exact surface type. The energy function is used in order to represent constraints of each stage and the minimum energy state is found using iterative method. Several experimental results show the generality of our approach and the execution speed of the proposed method is faster than that of a typical region merging method. This promises practical applications of our method.