Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud

Chikako TAKASAKI; Atsuko TAKEFUSA; Hidemoto NAKADA; Masato OGUCHI

doi:10.1587/transinf.2020DAP0009

Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud

Chikako TAKASAKI, Atsuko TAKEFUSA, Hidemoto NAKADA, Masato OGUCHI

Full Text Views

0

Cite this

Summary :

With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.

Publication: IEICE TRANSACTIONS on Information Vol.E104-D No.5 pp.539-550

Publication Date: 2021/05/01

Publicized: 2021/02/02

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2020DAP0009

Type of Manuscript: Special Section PAPER (Special Section on Data Engineering and Information Management)

Category

Authors

Chikako TAKASAKI
  Ochanomizu University
Atsuko TAKEFUSA
  National Institute of Informatics
Hidemoto NAKADA
  National Institute of Advanced Industrial Science and Technology (AIST)
Masato OGUCHI
  Ochanomizu University

Keyword

deep learning, machine learning, distributed processing, cloud computing, action recognition

Cite this

Copy

Chikako TAKASAKI, Atsuko TAKEFUSA, Hidemoto NAKADA, Masato OGUCHI, "Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud" in IEICE TRANSACTIONS on Information, vol. E104-D, no. 5, pp. 539-550, May 2021, doi: 10.1587/transinf.2020DAP0009.
Abstract: With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2020DAP0009/_p

Copy

@ARTICLE{e104-d_5_539,
author={Chikako TAKASAKI, Atsuko TAKEFUSA, Hidemoto NAKADA, Masato OGUCHI, },
journal={IEICE TRANSACTIONS on Information},
title={Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud},
year={2021},
volume={E104-D},
number={5},
pages={539-550},
abstract={With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.},
keywords={},
doi={10.1587/transinf.2020DAP0009},
ISSN={1745-1361},
month={May},}

Copy

TY - JOUR
TI - Action Recognition Using Pose Data in a Distributed Environment over the Edge and Cloud
T2 - IEICE TRANSACTIONS on Information
SP - 539
EP - 550
AU - Chikako TAKASAKI
AU - Atsuko TAKEFUSA
AU - Hidemoto NAKADA
AU - Masato OGUCHI
PY - 2021
DO - 10.1587/transinf.2020DAP0009
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E104-D
IS - 5
JA - IEICE TRANSACTIONS on Information
Y1 - May 2021
AB - With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.
ER -