Construction of an Efficient Divided/Distributed Neural Network Model Using Edge Computing

Ryuta SHINGAI; Yuria HIRAGA; Hisakazu FUKUOKA; Takamasa MITANI; Takashi NAKADA; Yasuhiko NAKASHIMA

doi:10.1587/transinf.2019EDP7326

Construction of an Efficient Divided/Distributed Neural Network Model Using Edge Computing

Ryuta SHINGAI, Yuria HIRAGA, Hisakazu FUKUOKA, Takamasa MITANI, Takashi NAKADA, Yasuhiko NAKASHIMA

Full Text Views

0

Cite this

Summary :

Modern deep learning has significantly improved performance and has been used in a wide variety of applications. Since the amount of computation required for the inference process of the neural network is large, it is processed not by the data acquisition location like a surveillance camera but by the server with abundant computing power installed in the data center. Edge computing is getting considerable attention to solve this problem. However, edge computing can provide limited computation resources. Therefore, we assumed a divided/distributed neural network model using both the edge device and the server. By processing part of the convolution layer on edge, the amount of communication becomes smaller than that of the sensor data. In this paper, we have evaluated AlexNet and the other eight models on the distributed environment and estimated FPS values with Wi-Fi, 3G, and 5G communication. To reduce communication costs, we also introduced the compression process before communication. This compression may degrade the object recognition accuracy. As necessary conditions, we set FPS to 30 or faster and object recognition accuracy to 69.7% or higher. This value is determined based on that of an approximation model that binarizes the activation of Neural Network. We constructed performance and energy models to find the optimal configuration that consumes minimum energy while satisfying the necessary conditions. Through the comprehensive evaluation, we found that the optimal configurations of all nine models. For small models, such as AlexNet, processing entire models in the edge was the best. On the other hand, for huge models, such as VGG16, processing entire models in the server was the best. For medium-size models, the distributed models were good candidates. We confirmed that our model found the most energy efficient configuration while satisfying FPS and accuracy requirements, and the distributed models successfully reduced the energy consumption up to 48.6%, and 6.6% on average. We also found that HEVC compression is important before transferring the input data or the feature data between the distributed inference processes.

Publication: IEICE TRANSACTIONS on Information Vol.E103-D No.10 pp.2072-2082

Publication Date: 2020/10/01

Publicized: 2020/07/02

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2019EDP7326

Type of Manuscript: PAPER

Category: Fundamentals of Information Systems

Authors

Ryuta SHINGAI
  Nara Institutet of Science and Technology
Yuria HIRAGA
  Nara Institutet of Science and Technology
Hisakazu FUKUOKA
  Nara Institutet of Science and Technology
Takamasa MITANI
  Nara Institutet of Science and Technology
Takashi NAKADA
  Nara Institutet of Science and Technology
Yasuhiko NAKASHIMA
  Nara Institutet of Science and Technology

Keyword

convolutional neural network, edge computing, distributed neural network, video compression

Cite this

Copy

Ryuta SHINGAI, Yuria HIRAGA, Hisakazu FUKUOKA, Takamasa MITANI, Takashi NAKADA, Yasuhiko NAKASHIMA, "Construction of an Efficient Divided/Distributed Neural Network Model Using Edge Computing" in IEICE TRANSACTIONS on Information, vol. E103-D, no. 10, pp. 2072-2082, October 2020, doi: 10.1587/transinf.2019EDP7326.
Abstract: Modern deep learning has significantly improved performance and has been used in a wide variety of applications. Since the amount of computation required for the inference process of the neural network is large, it is processed not by the data acquisition location like a surveillance camera but by the server with abundant computing power installed in the data center. Edge computing is getting considerable attention to solve this problem. However, edge computing can provide limited computation resources. Therefore, we assumed a divided/distributed neural network model using both the edge device and the server. By processing part of the convolution layer on edge, the amount of communication becomes smaller than that of the sensor data. In this paper, we have evaluated AlexNet and the other eight models on the distributed environment and estimated FPS values with Wi-Fi, 3G, and 5G communication. To reduce communication costs, we also introduced the compression process before communication. This compression may degrade the object recognition accuracy. As necessary conditions, we set FPS to 30 or faster and object recognition accuracy to 69.7% or higher. This value is determined based on that of an approximation model that binarizes the activation of Neural Network. We constructed performance and energy models to find the optimal configuration that consumes minimum energy while satisfying the necessary conditions. Through the comprehensive evaluation, we found that the optimal configurations of all nine models. For small models, such as AlexNet, processing entire models in the edge was the best. On the other hand, for huge models, such as VGG16, processing entire models in the server was the best. For medium-size models, the distributed models were good candidates. We confirmed that our model found the most energy efficient configuration while satisfying FPS and accuracy requirements, and the distributed models successfully reduced the energy consumption up to 48.6%, and 6.6% on average. We also found that HEVC compression is important before transferring the input data or the feature data between the distributed inference processes.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2019EDP7326/_p

Copy

@ARTICLE{e103-d_10_2072,
author={Ryuta SHINGAI, Yuria HIRAGA, Hisakazu FUKUOKA, Takamasa MITANI, Takashi NAKADA, Yasuhiko NAKASHIMA, },
journal={IEICE TRANSACTIONS on Information},
title={Construction of an Efficient Divided/Distributed Neural Network Model Using Edge Computing},
year={2020},
volume={E103-D},
number={10},
pages={2072-2082},
abstract={Modern deep learning has significantly improved performance and has been used in a wide variety of applications. Since the amount of computation required for the inference process of the neural network is large, it is processed not by the data acquisition location like a surveillance camera but by the server with abundant computing power installed in the data center. Edge computing is getting considerable attention to solve this problem. However, edge computing can provide limited computation resources. Therefore, we assumed a divided/distributed neural network model using both the edge device and the server. By processing part of the convolution layer on edge, the amount of communication becomes smaller than that of the sensor data. In this paper, we have evaluated AlexNet and the other eight models on the distributed environment and estimated FPS values with Wi-Fi, 3G, and 5G communication. To reduce communication costs, we also introduced the compression process before communication. This compression may degrade the object recognition accuracy. As necessary conditions, we set FPS to 30 or faster and object recognition accuracy to 69.7% or higher. This value is determined based on that of an approximation model that binarizes the activation of Neural Network. We constructed performance and energy models to find the optimal configuration that consumes minimum energy while satisfying the necessary conditions. Through the comprehensive evaluation, we found that the optimal configurations of all nine models. For small models, such as AlexNet, processing entire models in the edge was the best. On the other hand, for huge models, such as VGG16, processing entire models in the server was the best. For medium-size models, the distributed models were good candidates. We confirmed that our model found the most energy efficient configuration while satisfying FPS and accuracy requirements, and the distributed models successfully reduced the energy consumption up to 48.6%, and 6.6% on average. We also found that HEVC compression is important before transferring the input data or the feature data between the distributed inference processes.},
keywords={},
doi={10.1587/transinf.2019EDP7326},
ISSN={1745-1361},
month={October},}

Copy

TY - JOUR
TI - Construction of an Efficient Divided/Distributed Neural Network Model Using Edge Computing
T2 - IEICE TRANSACTIONS on Information
SP - 2072
EP - 2082
AU - Ryuta SHINGAI
AU - Yuria HIRAGA
AU - Hisakazu FUKUOKA
AU - Takamasa MITANI
AU - Takashi NAKADA
AU - Yasuhiko NAKASHIMA
PY - 2020
DO - 10.1587/transinf.2019EDP7326
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E103-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2020
AB - Modern deep learning has significantly improved performance and has been used in a wide variety of applications. Since the amount of computation required for the inference process of the neural network is large, it is processed not by the data acquisition location like a surveillance camera but by the server with abundant computing power installed in the data center. Edge computing is getting considerable attention to solve this problem. However, edge computing can provide limited computation resources. Therefore, we assumed a divided/distributed neural network model using both the edge device and the server. By processing part of the convolution layer on edge, the amount of communication becomes smaller than that of the sensor data. In this paper, we have evaluated AlexNet and the other eight models on the distributed environment and estimated FPS values with Wi-Fi, 3G, and 5G communication. To reduce communication costs, we also introduced the compression process before communication. This compression may degrade the object recognition accuracy. As necessary conditions, we set FPS to 30 or faster and object recognition accuracy to 69.7% or higher. This value is determined based on that of an approximation model that binarizes the activation of Neural Network. We constructed performance and energy models to find the optimal configuration that consumes minimum energy while satisfying the necessary conditions. Through the comprehensive evaluation, we found that the optimal configurations of all nine models. For small models, such as AlexNet, processing entire models in the edge was the best. On the other hand, for huge models, such as VGG16, processing entire models in the server was the best. For medium-size models, the distributed models were good candidates. We confirmed that our model found the most energy efficient configuration while satisfying FPS and accuracy requirements, and the distributed models successfully reduced the energy consumption up to 48.6%, and 6.6% on average. We also found that HEVC compression is important before transferring the input data or the feature data between the distributed inference processes.
ER -