An Area-Efficient Recurrent Neural Network Core for Unsupervised Time-Series Anomaly Detection

Takuya SAKUMA; Hiroki MATSUTANI

doi:10.1587/transele.2020LHP0003

IEICE TRANSACTIONS on Electronics

Open Access
An Area-Efficient Recurrent Neural Network Core for Unsupervised Time-Series Anomaly Detection

Takuya SAKUMA, Hiroki MATSUTANI

Full Text Views

33

Cite this

Free PDF (744.5KB)

Summary :

Since most sensor data depend on each other, time-series anomaly detection is one of practical applications of IoT devices. Such tasks are handled by Recurrent Neural Networks (RNNs) with a feedback structure, such as Long Short Term Memory. However, their learning phase based on Stochastic Gradient Descent (SGD) is computationally expensive for such edge devices. This issue is addressed by executing their learning on high-performance server machines, but it introduces a communication overhead and additional power consumption. On the other hand, Recursive Least-Squares Echo State Network (RLS-ESN) is a simple RNN that can be trained at low cost using the least-squares method rather than SGD. In this paper, we propose its area-efficient hardware implementation for edge devices and adapt it to human activity anomaly detection as an example of interdependent time-series sensor data. The model is implemented in Verilog HDL, synthesized with a 45 nm process technology, and evaluated in terms of the anomaly capability, hardware amount, and performance. The evaluation results demonstrate that the RLS-ESN core with a feedback structure is more robust to hyper parameters than an existing Online Sequential Extreme Learning Machine (OS-ELM) core. It consumes only 1.25 times larger hardware amount and 1.11 times longer latency than the existing OS-ELM core.

Publication: IEICE TRANSACTIONS on Electronics Vol.E104-C No.6 pp.247-256

Publication Date: 2021/06/01

Publicized: 2020/12/15

Online ISSN: 1745-1353

DOI: 10.1587/transele.2020LHP0003

Type of Manuscript: Special Section PAPER (Special Section on Low-Power and High-Speed Chips)

Category

Authors

Takuya SAKUMA
Keio University
Hiroki MATSUTANI
Keio University

Keyword

on-device learning, machine learning, anomaly detection

Cite this

Copy

Takuya SAKUMA, Hiroki MATSUTANI, "An Area-Efficient Recurrent Neural Network Core for Unsupervised Time-Series Anomaly Detection" in IEICE TRANSACTIONS on Electronics, vol. E104-C, no. 6, pp. 247-256, June 2021, doi: 10.1587/transele.2020LHP0003.
Abstract: Since most sensor data depend on each other, time-series anomaly detection is one of practical applications of IoT devices. Such tasks are handled by Recurrent Neural Networks (RNNs) with a feedback structure, such as Long Short Term Memory. However, their learning phase based on Stochastic Gradient Descent (SGD) is computationally expensive for such edge devices. This issue is addressed by executing their learning on high-performance server machines, but it introduces a communication overhead and additional power consumption. On the other hand, Recursive Least-Squares Echo State Network (RLS-ESN) is a simple RNN that can be trained at low cost using the least-squares method rather than SGD. In this paper, we propose its area-efficient hardware implementation for edge devices and adapt it to human activity anomaly detection as an example of interdependent time-series sensor data. The model is implemented in Verilog HDL, synthesized with a 45 nm process technology, and evaluated in terms of the anomaly capability, hardware amount, and performance. The evaluation results demonstrate that the RLS-ESN core with a feedback structure is more robust to hyper parameters than an existing Online Sequential Extreme Learning Machine (OS-ELM) core. It consumes only 1.25 times larger hardware amount and 1.11 times longer latency than the existing OS-ELM core.
URL: https://global.ieice.org/en_transactions/electronics/10.1587/transele.2020LHP0003/_p

Copy

@ARTICLE{e104-c_6_247,
author={Takuya SAKUMA, Hiroki MATSUTANI, },
journal={IEICE TRANSACTIONS on Electronics},
title={An Area-Efficient Recurrent Neural Network Core for Unsupervised Time-Series Anomaly Detection},
year={2021},
volume={E104-C},
number={6},
pages={247-256},
abstract={Since most sensor data depend on each other, time-series anomaly detection is one of practical applications of IoT devices. Such tasks are handled by Recurrent Neural Networks (RNNs) with a feedback structure, such as Long Short Term Memory. However, their learning phase based on Stochastic Gradient Descent (SGD) is computationally expensive for such edge devices. This issue is addressed by executing their learning on high-performance server machines, but it introduces a communication overhead and additional power consumption. On the other hand, Recursive Least-Squares Echo State Network (RLS-ESN) is a simple RNN that can be trained at low cost using the least-squares method rather than SGD. In this paper, we propose its area-efficient hardware implementation for edge devices and adapt it to human activity anomaly detection as an example of interdependent time-series sensor data. The model is implemented in Verilog HDL, synthesized with a 45 nm process technology, and evaluated in terms of the anomaly capability, hardware amount, and performance. The evaluation results demonstrate that the RLS-ESN core with a feedback structure is more robust to hyper parameters than an existing Online Sequential Extreme Learning Machine (OS-ELM) core. It consumes only 1.25 times larger hardware amount and 1.11 times longer latency than the existing OS-ELM core.},
keywords={},
doi={10.1587/transele.2020LHP0003},
ISSN={1745-1353},
month={June},}

Copy

TY - JOUR
TI - An Area-Efficient Recurrent Neural Network Core for Unsupervised Time-Series Anomaly Detection
T2 - IEICE TRANSACTIONS on Electronics
SP - 247
EP - 256
AU - Takuya SAKUMA
AU - Hiroki MATSUTANI
PY - 2021
DO - 10.1587/transele.2020LHP0003
JO - IEICE TRANSACTIONS on Electronics
SN - 1745-1353
VL - E104-C
IS - 6
JA - IEICE TRANSACTIONS on Electronics
Y1 - June 2021
AB - Since most sensor data depend on each other, time-series anomaly detection is one of practical applications of IoT devices. Such tasks are handled by Recurrent Neural Networks (RNNs) with a feedback structure, such as Long Short Term Memory. However, their learning phase based on Stochastic Gradient Descent (SGD) is computationally expensive for such edge devices. This issue is addressed by executing their learning on high-performance server machines, but it introduces a communication overhead and additional power consumption. On the other hand, Recursive Least-Squares Echo State Network (RLS-ESN) is a simple RNN that can be trained at low cost using the least-squares method rather than SGD. In this paper, we propose its area-efficient hardware implementation for edge devices and adapt it to human activity anomaly detection as an example of interdependent time-series sensor data. The model is implemented in Verilog HDL, synthesized with a 45 nm process technology, and evaluated in terms of the anomaly capability, hardware amount, and performance. The evaluation results demonstrate that the RLS-ESN core with a feedback structure is more robust to hyper parameters than an existing Online Sequential Extreme Learning Machine (OS-ELM) core. It consumes only 1.25 times larger hardware amount and 1.11 times longer latency than the existing OS-ELM core.
ER -

IEICE TRANSACTIONS on Electronics