We propose a method of interframe prediction in depth map coding that uses pixel-wise 3D motion estimated from encoded textures and depth maps. By using the 3D motion, an approximation of the depth map frame to be encoded is generated and used as a reference frame of block-wise motion compensation.

As an important extension of high-efficiency video coding (HEVC), screen content coding (SCC) includes various new coding modes, such as Intra Block Copy (IBC), Palette-based coding (Palette), and Adaptive Color Transform (ACT). These new tools have improved screen content encoding performance. This paper proposed a novel and fast algorithm by classifying Code Units (CUs) as text CUs or non-text CUs. For text CUs, the Intra mode was skipped in the compression process, whereas for non-text CUs, the IBC mode was skipped. The current CU depth range was then predicted according to its adjacent left CU depth level. Compared with the reference software HM16.7+SCM5.4, the proposed algorithm reduced encoding time by 23% on average and achieved an approximate 0.44% increase in Bjøntegaard delta bit rate and a negligible peak signal-to-noise ratio loss.

Depth-based action recognition has been attracting the attention of researchers because of the advantages of depth cameras over standard RGB cameras. One of these advantages is that depth data can provide richer information from multiple projections. In particular, multiple projections can be used to extract discriminative motion patterns that would not be discernible from one fixed projection. However, high computational costs have meant that recent studies have exploited only a small number of projections, such as front, side, and top. Thus, a large number of projections, which may be useful for discriminating actions, are discarded. In this paper, we propose an efficient method to exploit pools of multiple projections for recognizing actions in depth videos. First, we project 3D data onto multiple 2D-planes from different viewpoints sampled on a geodesic dome to obtain a large number of projections. Then, we train and test action classifiers independently for each projection. To reduce the computational cost, we propose a greedy method to select a small yet robust combination of projections. The idea is that best complementary projections will be considered first when searching for optimal combination. We conducted extensive experiments to verify the effectiveness of our method on three challenging benchmarks: MSR Action 3D, MSR Gesture 3D, and 3D Action Pairs. The experimental results show that our method outperforms other state-of-the-art methods while using a small number of projections.

A novel activation method for a B dopant implanted in a Si substrate using a soft X-ray undulator was examined. As the photon energy of the irradiated soft X-ray approached the energy of the core level of Si 2p, the activation ratio increased. The effect of soft X-ray irradiation on B activation was remarkable at temperatures lower than 400°C. The activation energy of B activation by soft X-ray irradiation (0.06 eV) was lower than that of B activation by furnace annealing (0.18 eV). The activation of the B dopant by soft X-ray irradiation occurs at low temperature, although the activation ratio shows small values of 6.2×10-3 at 110°C. The activation by soft X-ray is caused not only by thermal effects, but also electron excitation and atomic movement.

The the depth-of-field limitation of our eyes causes out-of-focus blur in the retinal images. The blur dynamically changes whenever we change our gaze and accordingly the scene point we are looking at changes its depth. This paper proposes an image display that reproduces retinal out-of-focus blur by using a stereoscopic display and eye trackers. Its purpose is to provide the viewer with more realistic visual experiences than conventional (stereoscopic) displays. Unlike previous similar systems that track only one of the viewer's eyes to estimate the gaze depth, the proposed system tracks both eyes individually using two eye trackers and estimates the gaze depth from the convergence angle calculated by triangulation. This provides several advantages over existing schemes, such as being able to deal with scenes having multiple depths. We describe detailed implementations of the proposed system and show the results of an experiment conducted to examine its effectiveness. In the experiment, creating a scene having two depths using two LCD displays together with a half mirror, we examined how difficult it is for viewers to distinguish between the real scene and its virtual reproduction created by the proposed display system. The results of the experiment show the effectiveness of the proposed approach.

The new generation video standard, i.e., High-efficiency Video Coding (HEVC), shows a significantly improved efficiency relative to the last standard, i.e., H.264. However, the quad tree structured coding units (CUs), which are adopted in HEVC to improve compression efficiency, cause high computational complexity. In this study, a novel fast algorithm is proposed for CU partition in intra coding to reduce the computational complexity. A rough minimum depth prediction of the largest CU method and an early termination method for CU partition based on the total coding bits of the current CU are employed. Many approaches have been proposed to reduce the encoding complexity of HEVC, but these methods do not use the total coding bits of the current CU as the main basis for judgment to judge the CU complexity. Compared with the reference software HM16.6, the proposed algorithm reduces encoding time by 45% on average and achieves an approximately 1.1% increase in Bjntegaard delta bit rate and a negligible peak signal-to-noise ratio loss.

In this article, we investigate the depth distribution and the depth spectra of linear codes over the ring R=F2+uF2+u2F2, where u3=1. By using homomorphism of abelian groups from R to F2 and the generator matrices of linear codes over R, the depth spectra of linear codes of type 8k14k22k3 are obtained. We also give the depth distribution of a linear code C over R. Finally, some examples are presented to illustrate our obtained results.

This paper proposes a foreground segmentation method for indoor environments using depth images only. It uses a morphological operator and histogram analysis to segment the foreground. In order to compare the accuracy for foreground segmentation, we use metric measurements of false positive rate (FPR), false negative rate (FNR), total error (TE), and a similarity measure (S). A series of experimental results using video sequences collected under various circumstances are discussed. The proposed system is also designed in a field-programmable gate array (FPGA) implementation with low hardware resources.

Displacement mapping has been widely used for adding geometric surface details to 3D mesh models. However, it requires sufficient tessellation of the mesh if fine details are to be represented. In this paper, we propose a method for applying the displacement mapping even on coarse models by using an augmented patch mesh. The patch mesh is a regularly tessellated flat square mesh, which is mapped onto the target area. Our method applies displacement mapping to the patch mesh for fitting it to the original mesh as well as for adding surface details. We generate a patch map, which stores three-dimensional displacements from the patch mesh to the original mesh. A displacement map is also provided for defining the new surface feature. The target area in the original mesh is then replaced with the patch mesh, and the patch mesh reconstructs the original shape using the patch map and the new surface detail is added using the displacement map. Our results show that our method conveniently adds surface features to various models. The proposed method is particularly useful if the surface features change dynamically since the original mesh is preserved and the separate patch mesh overwrites the target area at runtime.

We propose an unsharp-masking technique which preserves the hue of colors in images. This method magnifies the contrast of colors and spatially sharpens textures in images. The contrast magnification ratio is adaptively controlled. We show by experiments that this method enhances the color tone of photographs while keeping their perceptual scene depth.

In this paper, we present a computer vision-based human tracking system with multiple stereo cameras. Many widely used methods, such as KLT-tracker, update the trackers “frame-to-frame,” so that features extracted from one frame are utilized to update their current state. In contrast, we propose a novel optimization technique for the “multi-frame” approach that computes resultant trajectories directly from video sequences, in order to achieve high-level robustness against severe occlusion, which is known to be a challenging problem in computer vision. We developed a heuristic optimization technique to estimate human trajectories, instead of using dynamic programming (DP) or an iterative approach, which makes our method sufficiently computationally efficient to operate in realtime. Six video sequences where one to six people walk in a narrow laboratory space are processed using our system. The results confirm that our system is capable of tracking cluttered scenes in which severe occlusion occurs and people are frequently in close proximity to each other. Moreover, minimal information is required for tracking, instead of full camera images, which is communicated over the network. Hence, commonly used network devices are sufficient for constructing our tracking system.

In this paper, we propose a superpixel based depth map generation scheme for the application to monoscopic to stereoscopic video conversion. The proposed algorithm employs four main processes to generate depth maps for all frames in the video sequences. First, the depth maps of the key frames in the input sequence are generated by superpixel merging and some user interactions. Second, the frames in the input sequences are over-segmented by Simple Linear Iterative Clustering (SLIC) or depth aided SLIC method depending on whether or not they have the depth maps. Third, each superpixel in current frame is used to match the corresponding superpixel in its previous frame. Finally, depth map is propagated with a joint bilateral filter based on the estimated matching vector of each superpixel. We show an improved performance of the proposed algorithm through experimental results.

The development of 3D High Efficiency Video Coding (3D-HEVC) has resulted in a growing interest in the compression of depth-maps. To achieve better intra prediction performance, the Depth Modeling Mode (DMM) technique is employed as an intra prediction technique for depth-maps. However, the complexity and computation load have dramatically increased with the application of DMM. Therefore, in view of the limited colors in depth-maps, this paper presents a novel fast intra coding scheme based on Base Colors and Index Map (BCIM) to reduce the complexity of DMM effectively. Furthermore, the index map is remapped, and the Base Colors are coded by predictive coding in BCIM to improve compression efficiency. Compared with the intra prediction coding in DMM, the experimental results illustrate that the proposed scheme provides a decrease of approximately 51.2% in the intra prediction time. Meanwhile, the BD-rate increase is only 0.83% for the virtual intermediate views generated by Depth-Image-Based Rendering.

The authors have evaluated a method of expanding the bit depth of image signals called SGRAD, which requires fewer calculations, while degrading the sharpness of images less. Where noise is superimposed on image signals, the conventional method for obtaining high bit depth sometimes incorrectly detects the contours of images, making it unable to sufficiently correct the gradation. Requiring many line memories is also an issue with the conventional method when applying the process to vertical gradation. As a solution to this particular issue, SGRAD improves the method of detecting contours with transiting gradation to effectively correct the gradation of image signals which noise is superimposed on. In addition, the use of a prediction algorithm for detecting gradation reduces the scale of the circuit with less correction of the vertical gradation.

A novel depth perception control method for a monocular head-up display (HUD) in a car has been developed, which is called the dynamic perspective method. The method changes a size and a position of the HUD image such as arrow for depth perception and achieves a depth perception position of 120 [m] within an error of 30% in a simulation. However, it is difficult to achieve an accurate depth perception in the real world because of car vibration. To solve this problem, we focus on a property, namely, that people complement hidden images by previous continuously observed images. We hide the image on the HUD when the car is vibrated very much. We aim to point at the accurate depth position by using see-through HUD images while having users complement the hidden image positions based on the continuous images before car vibration. We developed a car that detects big vibration by an acceleration sensor and is equipped with our monocular HUD. Our new method pointed at the depth position more accurately than the previous method, which was confirmed by t-test.

We estimated the dependence of the perceived depth on luminance ratio by increasing the distance between the front and rear planes of a depth-fused 3-D (DFD) display. When the distance is great, the perceived depth has the tendency of nonlinear dependence on luminance ratio, which is very different from the almost linear dependence in a short-distance conventional DFD display. In a long-distance DFD display, the perceived depth is split to near the front plane at 0-40% of the rear luminance, near the rear plane at 70-100%, and the midpoint of the front and rear planes at 40-60%. Thus, the luminance-ratio dependence of perceived depth changes widely with the distance.

This letter proposes a novel depth-guided inpainting scheme for the high quality hole-filling in 2D-to-3D video conversion. The proposed scheme detects and removes foreground depth layers in an image patch, enabling appropriate patch formation using only disoccluded background information. This background only patch formation helps to avoid the propagation of wrong depths over hole area, and thus improve the overall quality of converted 3D video experience. Experimental results demonstrate the proposed scheme provides visually much more pleasing inpainting results with better preserved object edges compared to the state-of-the-art depth-guided inpainting schemes.

A method of frame synchronization between the color video and depth-map video for depth based 3D video using edge coherence is proposed. We find a synchronized pair of frames using edge coherence by computing the maximum number of overlapped edge pixels between the color video and depth-map video in regions of temporal frame difference. The experimental results show that the proposed method can be used for synchronization of depth-based 3D video and that it is robust against Gaussian noise with σ = less than 30 and video compression by H.264/AVC with QP = less than 44.

A specification for digital cinema systems which deal with movies digitally from production to delivery as well as projection on the screens is recommended by DCI (Digital Cinema Initiative), and the systems based on this specification have already been developed and installed in theaters. The parameters of the systems that play an important role in determining image quality include image resolution, quantization bit depth, color space, gamma characteristics, and data compression methods. This paper comparatively discusses a relation between required bit depth and gamma quantization using both of a human visual system for grayscale images and two color difference models for color images. The required bit depth obtained from a contrast sensitivity function against grayscale images monotonically decreases as the gamma value increases, while it has a minimum value when the gamma is 2.9 to 3.0 from both of the CIE 1976 L* a* b* and CIEDE2000 color difference models. It is also shown that the bit depth derived from the contrast sensitivity function is one bit greater than that derived from the color difference models at the gamma value of 2.6. Moreover, a comparison between the color differences computed with the CIE 1976 L* a* b* and CIEDE2000 leads to a same result from the view point of the required bit depth for digital cinema systems.

Images captured under foggy conditions often exhibit poor contrast and color. This is primarily due to the air-light which degrades image quality exponentially with fog depth between the scene and the camera. In this paper, we restore fog-degraded images by first estimating depth using the physical model characterizing the RGB channels in a single monocular image. The fog effects are then removed by subtracting the estimated irradiance, which is empirically related to the scene depth information obtained, from the total irradiance received by the sensor. Effective restoration of color and contrast of images taken under foggy conditions are demonstrated. In the experiments, we validate the effectiveness of our method compared with conventional method.