Deep-Learning-Assisted Single-Pixel Imaging for Gesture Recognition in Consideration of Privacy

Naoya MUKOJIMA; Masaki YASUGI; Yasuhiro MIZUTANI; Takeshi YASUI; Hirotsugu YAMAMOTO

doi:10.1587/transele.2021DII0002

IEICE TRANSACTIONS on Electronics

Open Access
Deep-Learning-Assisted Single-Pixel Imaging for Gesture Recognition in Consideration of Privacy

Naoya MUKOJIMA, Masaki YASUGI, Yasuhiro MIZUTANI, Takeshi YASUI, Hirotsugu YAMAMOTO

Full Text Views

66

Cite this

Free PDF (1.3MB)

Summary :

We have utilized single-pixel imaging and deep-learning to solve the privacy-preserving problem in gesture recognition for interactive display. Silhouette images of hand gestures were acquired by use of a display panel as an illumination. Reconstructions of gesture images have been performed by numerical experiments on single-pixel imaging by changing the number of illumination mask patterns. For the training and the image restoration with deep learning, we prepared reconstructed data with 250 and 500 illuminations as datasets. For each of the 250 and 500 illuminations, we prepared 9000 datasets in which original images and reconstructed data were paired. Of these data, 8500 data were used for training a neural network (6800 data for training and 1700 data for validation), and 500 data were used to evaluate the accuracy of image restoration. Our neural network, based on U-net, was able to restore images close to the original images even from reconstructed data with greatly reduced number of illuminations, which is 1/40 of the single-pixel imaging without deep learning. Compared restoration accuracy between cases using shadowgraph (black on white background) and negative-positive reversed images (white on black background) as silhouette image, the accuracy of the restored image was lower for negative-positive-reversed images when the number of illuminations was small. Moreover, we found that the restoration accuracy decreased in the order of rock, scissor, and paper. Shadowgraph is suitable for gesture silhouette, and it is necessary to prepare training data and construct neural networks, to avoid the restoration accuracy between gestures when further reducing the number of illuminations.

Publication: IEICE TRANSACTIONS on Electronics Vol.E105-C No.2 pp.79-85

Publication Date: 2022/02/01

Publicized: 2021/08/17

Online ISSN: 1745-1353

DOI: 10.1587/transele.2021DII0002

Type of Manuscript: Special Section INVITED PAPER (Special Section on Electronic Displays)

Category

Authors

Naoya MUKOJIMA
  Utsunomiya Univ.
Masaki YASUGI
  Utsunomiya Univ.
Yasuhiro MIZUTANI
  Osaka Univ.
Takeshi YASUI
  Tokushima Univ.
Hirotsugu YAMAMOTO
  Utsunomiya Univ.

Keyword

single-pixel imaging, deep-learning, gesture recognition, U-net, privacy-preserving

Cite this

Copy

Naoya MUKOJIMA, Masaki YASUGI, Yasuhiro MIZUTANI, Takeshi YASUI, Hirotsugu YAMAMOTO, "Deep-Learning-Assisted Single-Pixel Imaging for Gesture Recognition in Consideration of Privacy" in IEICE TRANSACTIONS on Electronics, vol. E105-C, no. 2, pp. 79-85, February 2022, doi: 10.1587/transele.2021DII0002.
Abstract: We have utilized single-pixel imaging and deep-learning to solve the privacy-preserving problem in gesture recognition for interactive display. Silhouette images of hand gestures were acquired by use of a display panel as an illumination. Reconstructions of gesture images have been performed by numerical experiments on single-pixel imaging by changing the number of illumination mask patterns. For the training and the image restoration with deep learning, we prepared reconstructed data with 250 and 500 illuminations as datasets. For each of the 250 and 500 illuminations, we prepared 9000 datasets in which original images and reconstructed data were paired. Of these data, 8500 data were used for training a neural network (6800 data for training and 1700 data for validation), and 500 data were used to evaluate the accuracy of image restoration. Our neural network, based on U-net, was able to restore images close to the original images even from reconstructed data with greatly reduced number of illuminations, which is 1/40 of the single-pixel imaging without deep learning. Compared restoration accuracy between cases using shadowgraph (black on white background) and negative-positive reversed images (white on black background) as silhouette image, the accuracy of the restored image was lower for negative-positive-reversed images when the number of illuminations was small. Moreover, we found that the restoration accuracy decreased in the order of rock, scissor, and paper. Shadowgraph is suitable for gesture silhouette, and it is necessary to prepare training data and construct neural networks, to avoid the restoration accuracy between gestures when further reducing the number of illuminations.
URL: https://global.ieice.org/en_transactions/electronics/10.1587/transele.2021DII0002/_p

Copy

@ARTICLE{e105-c_2_79,
author={Naoya MUKOJIMA, Masaki YASUGI, Yasuhiro MIZUTANI, Takeshi YASUI, Hirotsugu YAMAMOTO, },
journal={IEICE TRANSACTIONS on Electronics},
title={Deep-Learning-Assisted Single-Pixel Imaging for Gesture Recognition in Consideration of Privacy},
year={2022},
volume={E105-C},
number={2},
pages={79-85},
abstract={We have utilized single-pixel imaging and deep-learning to solve the privacy-preserving problem in gesture recognition for interactive display. Silhouette images of hand gestures were acquired by use of a display panel as an illumination. Reconstructions of gesture images have been performed by numerical experiments on single-pixel imaging by changing the number of illumination mask patterns. For the training and the image restoration with deep learning, we prepared reconstructed data with 250 and 500 illuminations as datasets. For each of the 250 and 500 illuminations, we prepared 9000 datasets in which original images and reconstructed data were paired. Of these data, 8500 data were used for training a neural network (6800 data for training and 1700 data for validation), and 500 data were used to evaluate the accuracy of image restoration. Our neural network, based on U-net, was able to restore images close to the original images even from reconstructed data with greatly reduced number of illuminations, which is 1/40 of the single-pixel imaging without deep learning. Compared restoration accuracy between cases using shadowgraph (black on white background) and negative-positive reversed images (white on black background) as silhouette image, the accuracy of the restored image was lower for negative-positive-reversed images when the number of illuminations was small. Moreover, we found that the restoration accuracy decreased in the order of rock, scissor, and paper. Shadowgraph is suitable for gesture silhouette, and it is necessary to prepare training data and construct neural networks, to avoid the restoration accuracy between gestures when further reducing the number of illuminations.},
keywords={},
doi={10.1587/transele.2021DII0002},
ISSN={1745-1353},
month={February},}

Copy

TY - JOUR
TI - Deep-Learning-Assisted Single-Pixel Imaging for Gesture Recognition in Consideration of Privacy
T2 - IEICE TRANSACTIONS on Electronics
SP - 79
EP - 85
AU - Naoya MUKOJIMA
AU - Masaki YASUGI
AU - Yasuhiro MIZUTANI
AU - Takeshi YASUI
AU - Hirotsugu YAMAMOTO
PY - 2022
DO - 10.1587/transele.2021DII0002
JO - IEICE TRANSACTIONS on Electronics
SN - 1745-1353
VL - E105-C
IS - 2
JA - IEICE TRANSACTIONS on Electronics
Y1 - February 2022
AB - We have utilized single-pixel imaging and deep-learning to solve the privacy-preserving problem in gesture recognition for interactive display. Silhouette images of hand gestures were acquired by use of a display panel as an illumination. Reconstructions of gesture images have been performed by numerical experiments on single-pixel imaging by changing the number of illumination mask patterns. For the training and the image restoration with deep learning, we prepared reconstructed data with 250 and 500 illuminations as datasets. For each of the 250 and 500 illuminations, we prepared 9000 datasets in which original images and reconstructed data were paired. Of these data, 8500 data were used for training a neural network (6800 data for training and 1700 data for validation), and 500 data were used to evaluate the accuracy of image restoration. Our neural network, based on U-net, was able to restore images close to the original images even from reconstructed data with greatly reduced number of illuminations, which is 1/40 of the single-pixel imaging without deep learning. Compared restoration accuracy between cases using shadowgraph (black on white background) and negative-positive reversed images (white on black background) as silhouette image, the accuracy of the restored image was lower for negative-positive-reversed images when the number of illuminations was small. Moreover, we found that the restoration accuracy decreased in the order of rock, scissor, and paper. Shadowgraph is suitable for gesture silhouette, and it is necessary to prepare training data and construct neural networks, to avoid the restoration accuracy between gestures when further reducing the number of illuminations.
ER -

IEICE TRANSACTIONS on Electronics