High-quality depth images are required for stable and accurate computer vision. Depth images captured by depth cameras tend to be noisy, incomplete, and of low-resolution. Therefore, increasing the accuracy and resolution of depth images is desirable. We propose a method for reducing the noise and holes from depth images pixel by pixel, and increasing resolution. For each pixel in the target image, the linear space from the focal point of the camera through each pixel to the existing object is divided into equally spaced grids. In each grid, the difference from each grid to the object surface is obtained from multiple tracked depth images, which have noisy depth values of the respective image pixels. Then, the coordinates of the correct object surface are obtainable by reducing the depth random noise. The missing values are completed. The resolution can also be increased by creating new pixels between existing pixels and by then using the same process as that used for noise reduction. Evaluation results have demonstrated that the proposed method can do processing with less GPU memory. Furthermore, the proposed method was able to reduce noise more accurately, especially around edges, and was able to process more details of objects than the conventional method. The super-resolution of the proposed method also produced a high-resolution depth image with smoother and more accurate edges than the conventional methods.
Masahiro MURAYAMA
Kyoto University
Toyohiro HIGASHIYAMA
Nomura Research Institute, Ltd.
Yuki HARAZONO
Kyoto University
Hirotake ISHII
Kyoto University
Hiroshi SHIMODA
Kyoto University
Shinobu OKIDO
Hitachi--GE Nuclear Energy, Ltd.
Yasuyoshi TARUTA
Japan Atomic Energy Agency
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Copy
Masahiro MURAYAMA, Toyohiro HIGASHIYAMA, Yuki HARAZONO, Hirotake ISHII, Hiroshi SHIMODA, Shinobu OKIDO, Yasuyoshi TARUTA, "Depth Image Noise Reduction and Super-Resolution by Pixel-Wise Multi-Frame Fusion" in IEICE TRANSACTIONS on Information,
vol. E105-D, no. 6, pp. 1211-1224, June 2022, doi: 10.1587/transinf.2021EDP7226.
Abstract: High-quality depth images are required for stable and accurate computer vision. Depth images captured by depth cameras tend to be noisy, incomplete, and of low-resolution. Therefore, increasing the accuracy and resolution of depth images is desirable. We propose a method for reducing the noise and holes from depth images pixel by pixel, and increasing resolution. For each pixel in the target image, the linear space from the focal point of the camera through each pixel to the existing object is divided into equally spaced grids. In each grid, the difference from each grid to the object surface is obtained from multiple tracked depth images, which have noisy depth values of the respective image pixels. Then, the coordinates of the correct object surface are obtainable by reducing the depth random noise. The missing values are completed. The resolution can also be increased by creating new pixels between existing pixels and by then using the same process as that used for noise reduction. Evaluation results have demonstrated that the proposed method can do processing with less GPU memory. Furthermore, the proposed method was able to reduce noise more accurately, especially around edges, and was able to process more details of objects than the conventional method. The super-resolution of the proposed method also produced a high-resolution depth image with smoother and more accurate edges than the conventional methods.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2021EDP7226/_p
Copy
@ARTICLE{e105-d_6_1211,
author={Masahiro MURAYAMA, Toyohiro HIGASHIYAMA, Yuki HARAZONO, Hirotake ISHII, Hiroshi SHIMODA, Shinobu OKIDO, Yasuyoshi TARUTA, },
journal={IEICE TRANSACTIONS on Information},
title={Depth Image Noise Reduction and Super-Resolution by Pixel-Wise Multi-Frame Fusion},
year={2022},
volume={E105-D},
number={6},
pages={1211-1224},
abstract={High-quality depth images are required for stable and accurate computer vision. Depth images captured by depth cameras tend to be noisy, incomplete, and of low-resolution. Therefore, increasing the accuracy and resolution of depth images is desirable. We propose a method for reducing the noise and holes from depth images pixel by pixel, and increasing resolution. For each pixel in the target image, the linear space from the focal point of the camera through each pixel to the existing object is divided into equally spaced grids. In each grid, the difference from each grid to the object surface is obtained from multiple tracked depth images, which have noisy depth values of the respective image pixels. Then, the coordinates of the correct object surface are obtainable by reducing the depth random noise. The missing values are completed. The resolution can also be increased by creating new pixels between existing pixels and by then using the same process as that used for noise reduction. Evaluation results have demonstrated that the proposed method can do processing with less GPU memory. Furthermore, the proposed method was able to reduce noise more accurately, especially around edges, and was able to process more details of objects than the conventional method. The super-resolution of the proposed method also produced a high-resolution depth image with smoother and more accurate edges than the conventional methods.},
keywords={},
doi={10.1587/transinf.2021EDP7226},
ISSN={1745-1361},
month={June},}
Copy
TY - JOUR
TI - Depth Image Noise Reduction and Super-Resolution by Pixel-Wise Multi-Frame Fusion
T2 - IEICE TRANSACTIONS on Information
SP - 1211
EP - 1224
AU - Masahiro MURAYAMA
AU - Toyohiro HIGASHIYAMA
AU - Yuki HARAZONO
AU - Hirotake ISHII
AU - Hiroshi SHIMODA
AU - Shinobu OKIDO
AU - Yasuyoshi TARUTA
PY - 2022
DO - 10.1587/transinf.2021EDP7226
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E105-D
IS - 6
JA - IEICE TRANSACTIONS on Information
Y1 - June 2022
AB - High-quality depth images are required for stable and accurate computer vision. Depth images captured by depth cameras tend to be noisy, incomplete, and of low-resolution. Therefore, increasing the accuracy and resolution of depth images is desirable. We propose a method for reducing the noise and holes from depth images pixel by pixel, and increasing resolution. For each pixel in the target image, the linear space from the focal point of the camera through each pixel to the existing object is divided into equally spaced grids. In each grid, the difference from each grid to the object surface is obtained from multiple tracked depth images, which have noisy depth values of the respective image pixels. Then, the coordinates of the correct object surface are obtainable by reducing the depth random noise. The missing values are completed. The resolution can also be increased by creating new pixels between existing pixels and by then using the same process as that used for noise reduction. Evaluation results have demonstrated that the proposed method can do processing with less GPU memory. Furthermore, the proposed method was able to reduce noise more accurately, especially around edges, and was able to process more details of objects than the conventional method. The super-resolution of the proposed method also produced a high-resolution depth image with smoother and more accurate edges than the conventional methods.
ER -