This paper presents a new hole-filling method that uses extrapolated spatio-temporal background information to obtain a synthesized free-view. A new background codebook for extracting reliable temporal background information is introduced. In addition, the paper addresses estimating spatial local background to distinguish background and foreground regions so that spatial background information can be extrapolated. Background holes are filled by combining spatial and temporal background information. Finally, exemplar-based inpainting is applied to fill in the remaining holes using a new priority function. The experimental results demonstrated that satisfactory synthesized views can be obtained using the proposed algorithm.

Multi-view video can provide users with three-dimensional (3-D) and virtual reality perception through multiple viewing angles. In recent years, depth image-based rendering (DIBR) has been generally used to synthesize virtual view images in free viewpoint television (FTV) and 3-D video. To conceal the zero-region more accurately and improve the quality of a virtual view synthesized frame, an integrated hole-filling algorithm for view synthesis is proposed in this paper. The proposed algorithm contains five parts: an algorithm for distinguishing different regions, foreground and background boundary detection, texture image isophotes detection, a textural and structural isophote prediction algorithm, and an in-painting algorithm with gradient priority order. Based on the texture isophote prediction with a geometrical principle and the in-painting algorithm with a gradient priority order, the boundary information of the foreground is considerably clearer and the texture information in the zero-region can be concealed much more accurately than in previous works. The vision quality mainly depends on the distortion of the structural information. Experimental results indicate that the proposed algorithm improves not only the objective quality of the virtual image, but also its subjective quality considerably; human vision is also clearly improved based on the subjective results. In particular, the algorithm ensures the boundary contours of the foreground objects and the textural and structural information.

Surface integrity of 3D medical data is crucial for surgery simulation or virtual diagnoses. However, undesirable holes often exist due to external damage on bodies or accessibility limitation on scanners. To bridge the gap, hole-filling for medical imaging is a popular research topic in recent years [1]-[3]. Considering that a medical image, e.g. CT or MRI, has the natural form of a tensor, we recognize the problem of medical hole-filling as the extension of Principal Component Pursuit (PCP) problem from matrix case to tensor case. Since the new problem in the tensor case is much more difficult than the matrix case, an efficient algorithm for the extension is presented by relaxation technique. The most significant feature of our algorithm is that unlike traditional methods which follow a strictly local approach, our method fixes the hole by the global structure in the specific medical data. Another important difference from the previous algorithm [4] is that our algorithm is able to automatically separate the completed data from the hole in an implicit manner. Our experiments demonstrate that the proposed method can lead to satisfactory results.