Due to the global outbreak of coronaviruses, people are increasingly wearing masks even when photographed. As a result, photos uploaded to web pages and social networking services with the lower half of the face hidden are less likely to convey the attractiveness of the photographed persons. In this study, we propose a method to complete facial mask regions using StyleGAN2, a type of Generative Adversarial Networks (GAN). In the proposed method, a reference image of the same person without a mask is prepared separately from a target image of the person wearing a mask. After the mask region in the target image is temporarily inpainted, the face orientation and contour of the person in the reference image are changed to match those of the target image using StyleGAN2. The changed image is then composited into the mask region while correcting the color tone to produce a mask-free image while preserving the person's features.

This paper proposes an image inpainting algorithm based on multiple linear models and matrix rank minimization. Several inpainting algorithms have been previously proposed based on the assumption that an image can be modeled using autoregressive (AR) models. However, these algorithms perform poorly when applied to natural photographs because they assume that an image is modeled by a position-invariant linear model with a fixed model order. In order to improve inpainting quality, this work introduces a multiple AR model and proposes an image inpainting algorithm based on multiple matrix rank minimization with sparse regularization. In doing so, a practical algorithm is provided based on the iterative partial matrix shrinkage algorithm, with numerical examples showing the effectiveness of the proposed algorithm.

This letter presents an objective quality index for benchmarking image inpainting algorithms. Under the guidance of the masks of damaged areas, the boundary region and the inpainting region are first located. Then, the statistical features are extracted from the boundary and inpainting regions respectively. For the boundary region, we utilize Weibull distribution to fit the gradient magnitude histograms of the exterior and interior regions around the boundary, and the Kullback-Leibler Divergence (KLD) is calculated to measure the boundary distortions caused by imperfect inpainting. Meanwhile, the quality of the inpainting region is measured by comparing the naturalness factors between the inpainted image and the reference image. Experimental results demonstrate that the proposed metric outperforms the relevant state-of-the-art quality metrics.

This paper proposes an algorithm for exemplar-based image inpainting, which produces the same result as that of Criminisi's original scheme but at the cost of much smaller computation cost. The idea is to compute mean and standard deviation of every patch in the image, and use the values to decide whether to carry out pixel by pixel comparison or not when searching for the best matching patch. Due to the missing pixels in the target patch, the same pixels in the candidate patch should be omitted when computing the distance between patches. Thus, we first compute the range of mean and standard deviation of a candidate patch with missing pixels, using the average and standard deviation of the entire patch. Then we use the range to determine if the pixel comparison should be conducted. Measurements with well-known images in the inpainting literature show that the algorithm can save significant amount of computation cost, without risking degradation of image quality.

This letter presents a scheme to improve the running time of exemplar-based image inpainting, first proposed by Criminisi et al. In the exemplar-based image inpainting, a patch that contains unknown pixels is compared to all the patches in the known region in order to find the best match. This is very time-consuming and hinders the practicality of Criminisi's method to be used in real time. We show that a simple bounding algorithm can significantly reduce number of distance calculations, and thus the running time. Performance of the bounding algorithm is affected by the order of patches that are compared, as well as the order of pixels in a patch. We present pixel and patch ordering schemes that improve the performance of bounding algorithms. Experiments with well-known images used in inpainting literature show that the proposed reordering scheme can reduce running time of the bounding algorithm up to 50%.

This paper presents a novel inpainting method based on structure estimation. The method first estimates an initial image that captures the rough structure and colors in the missing region. This image is generated by probabilistically estimating the gradient within the missing region based on edge segments intersecting its boundary, and then by flooding the colors on the boundary into the missing region. The color flooding is formulated as an energy minimization problem, and is efficiently optimized by the conjugate gradient method. Finally, by locally replacing the missing region with local patches similar to both the adjacent patches and the initial image, the inpainted image is synthesized. The initial image not only serves as a guide to ensure the underlying structure is preserved, but also allows the patch selection process to be carried out in a greedy manner, which leads to substantial speedup. Experimental results show the proposed method is capable of preserving the underlying structure in the missing region, while achieving more than 5 times faster computational speed than the state-of-the-art inpainting method. Subjective evaluation of image quality also shows the proposed method outperforms the previous methods.

In this letter, a novel adaptive total variation (ATV) model is proposed for image inpainting. The classical TV model is a partial differential equation (PDE)-based technique. While the TV model can preserve the image edges well, it has some drawbacks, such as staircase effect in the inpainted image and slow convergence rate. By analyzing the diffusion mechanism of TV model and introducing a new edge detection operator named difference curvature, we propose a novel ATV inpainting model. The proposed ATV model can diffuse the image information smoothly and quickly, namely, this model not only eliminates the staircase effect but also accelerates the convergence rate. Experimental results demonstrate the effectiveness of the proposed scheme.

This paper proposes a novel image inpainting method based on bandelet transform. This technique is based on a multi-resolution layer to perform image restoration, and mainly utilizes the geometrical flow of the neighboring texture of the damaged regions as the basis of restoration. By performing the warp transform with geometrical flows, it transforms the textural variation into the nearing domain axis utilizing the bandelet decomposition method to decompose the non-relative textures into different bands, and then combines them with the affine search method to perform image restoration. The experimental results show that the proposed method can simplify the complexity of the repair decision method and improve the quality of HVS, and thus, repaired results to contain the image of contour of high change, and in addition, offer a texture image of high-frequency variation. These repair results can lead to state-of-the-art results.

Currently, the automatic image inpainting methods emphasize the inpainting techniques either globally or locally. They didn't consider the merits of global and local techniques to compensate each other. On the contrary, the artists fixed an image in global view firstly, and then focus on the local features of it, when they repaired it. This paper proposes a progressive processing of image inpainting method based on multi-resolution analysis. In damaged and defective area, we imitate the artistic techniques to approach the effectiveness of image inpainting in human vision. First, we use the multi-resolution characteristics of wavelet transform, from the lowest spatial-frequency layer to the higher one, to analyze the image from global-area to local-area progressively. Then, we utilize the variance of the energy of wavelet coefficients within each image block, to decide the priority of inpainting blocks. Finally, we extract the multi-resolution features of each block. We take account of the correlation among horizontal, vertical and diagonal directions, to determine the inpainting strategy for filling image pixels and approximate a high-quality image inpainting to human vision. In our experiments, the performance of the proposed method is superior to the existing methods.