IEICE TRANSACTIONS on Information
Speeding up Deep Neural Networks in Speech Recognition with Piecewise Quantized Sigmoidal Activation Function
Summary :
This paper proposes a new quantization framework on activation function of deep neural networks (DNN). We implement fixed-point DNN by quantizing the activations into powers-of-two integers. The costly multiplication operations in using DNN can be replaced with low-cost bit-shifts to massively save computations. Thus, applying DNN-based speech recognition on embedded systems becomes much easier. Experiments show that the proposed method leads to no performance degradation.
- Publication
- IEICE TRANSACTIONS on Information Vol.E99-D No.10 pp.2558-2561
- Publication Date
- 2016/10/01
- Publicized
- 2016/07/19
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2016SLL0007
- Type of Manuscript
- Special Section LETTER (Special Section on Recent Advances in Machine Learning for Spoken Language Processing)
- Category
- Acoustic modeling
Authors
Anhao XING
Chinese Academy of Sciences
Qingwei ZHAO
Chinese Academy of Sciences
Yonghong YAN
Chinese Academy of Sciences,Xinjiang Laboratory of Minority Speech and Language Information Processing
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Anhao XING, Qingwei ZHAO, Yonghong YAN, "Speeding up Deep Neural Networks in Speech Recognition with Piecewise Quantized Sigmoidal Activation Function" in IEICE TRANSACTIONS on Information,
vol. E99-D, no. 10, pp. 2558-2561, October 2016, doi: 10.1587/transinf.2016SLL0007.
Abstract: This paper proposes a new quantization framework on activation function of deep neural networks (DNN). We implement fixed-point DNN by quantizing the activations into powers-of-two integers. The costly multiplication operations in using DNN can be replaced with low-cost bit-shifts to massively save computations. Thus, applying DNN-based speech recognition on embedded systems becomes much easier. Experiments show that the proposed method leads to no performance degradation.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2016SLL0007/_p
Copy
@ARTICLE{e99-d_10_2558,
author={Anhao XING, Qingwei ZHAO, Yonghong YAN, },
journal={IEICE TRANSACTIONS on Information},
title={Speeding up Deep Neural Networks in Speech Recognition with Piecewise Quantized Sigmoidal Activation Function},
year={2016},
volume={E99-D},
number={10},
pages={2558-2561},
abstract={This paper proposes a new quantization framework on activation function of deep neural networks (DNN). We implement fixed-point DNN by quantizing the activations into powers-of-two integers. The costly multiplication operations in using DNN can be replaced with low-cost bit-shifts to massively save computations. Thus, applying DNN-based speech recognition on embedded systems becomes much easier. Experiments show that the proposed method leads to no performance degradation.},
keywords={},
doi={10.1587/transinf.2016SLL0007},
ISSN={1745-1361},
month={October},}
Copy
TY - JOUR
TI - Speeding up Deep Neural Networks in Speech Recognition with Piecewise Quantized Sigmoidal Activation Function
T2 - IEICE TRANSACTIONS on Information
SP - 2558
EP - 2561
AU - Anhao XING
AU - Qingwei ZHAO
AU - Yonghong YAN
PY - 2016
DO - 10.1587/transinf.2016SLL0007
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E99-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2016
AB - This paper proposes a new quantization framework on activation function of deep neural networks (DNN). We implement fixed-point DNN by quantizing the activations into powers-of-two integers. The costly multiplication operations in using DNN can be replaced with low-cost bit-shifts to massively save computations. Thus, applying DNN-based speech recognition on embedded systems becomes much easier. Experiments show that the proposed method leads to no performance degradation.
ER -
English
