On a Weight Limit Approach for Enhancing Fault Tolerance of Feedforward Neural Networks
Summary :
To enhance fault tolerance ability of the feedforward neural networks (NNs for short) implemented in hardware, we discuss the learning algorithm that converges without adding extra neurons and a large amount of extra learning time and cycles. Our algorithm modified from the standard backpropagation algorithm (SBPA for short) limits synaptic weights of neurons in range during learning phase. The upper and lower bounds of the weights are calculated according to the average and standard deviation of them. Then our algorithm reupdates any weight beyond the calculated range to the upper or lower bound. Since the above enables us to decrease the standard deviation of the weights, it is useful in enhancing fault tolerance. We apply NNs trained with other algorithms and our one to a character recognition problem. It is shown that our one is superior to other ones in reliability, extra learning time and/or extra learning cycles. Besides we clarify that our algorithm never degrades the generalization ability of NNs although it coerces the weights within the calculated range.
- Publication
- IEICE TRANSACTIONS on Information Vol.E83-D No.11 pp.1931-1939
- Publication Date
- 2000/11/25
- Publicized
- Online ISSN
- DOI
- Type of Manuscript
- PAPER
- Category
- Fault Tolerance
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Naotake KAMIURA, Teijiro ISOKAWA, Yutaka HATA, Nobuyuki MATSUI, Kazuharu YAMATO, "On a Weight Limit Approach for Enhancing Fault Tolerance of Feedforward Neural Networks" in IEICE TRANSACTIONS on Information,
vol. E83-D, no. 11, pp. 1931-1939, November 2000, doi: .
Abstract: To enhance fault tolerance ability of the feedforward neural networks (NNs for short) implemented in hardware, we discuss the learning algorithm that converges without adding extra neurons and a large amount of extra learning time and cycles. Our algorithm modified from the standard backpropagation algorithm (SBPA for short) limits synaptic weights of neurons in range during learning phase. The upper and lower bounds of the weights are calculated according to the average and standard deviation of them. Then our algorithm reupdates any weight beyond the calculated range to the upper or lower bound. Since the above enables us to decrease the standard deviation of the weights, it is useful in enhancing fault tolerance. We apply NNs trained with other algorithms and our one to a character recognition problem. It is shown that our one is superior to other ones in reliability, extra learning time and/or extra learning cycles. Besides we clarify that our algorithm never degrades the generalization ability of NNs although it coerces the weights within the calculated range.
URL: https://global.ieice.org/en_transactions/information/10.1587/e83-d_11_1931/_p
Copy
@ARTICLE{e83-d_11_1931,
author={Naotake KAMIURA, Teijiro ISOKAWA, Yutaka HATA, Nobuyuki MATSUI, Kazuharu YAMATO, },
journal={IEICE TRANSACTIONS on Information},
title={On a Weight Limit Approach for Enhancing Fault Tolerance of Feedforward Neural Networks},
year={2000},
volume={E83-D},
number={11},
pages={1931-1939},
abstract={To enhance fault tolerance ability of the feedforward neural networks (NNs for short) implemented in hardware, we discuss the learning algorithm that converges without adding extra neurons and a large amount of extra learning time and cycles. Our algorithm modified from the standard backpropagation algorithm (SBPA for short) limits synaptic weights of neurons in range during learning phase. The upper and lower bounds of the weights are calculated according to the average and standard deviation of them. Then our algorithm reupdates any weight beyond the calculated range to the upper or lower bound. Since the above enables us to decrease the standard deviation of the weights, it is useful in enhancing fault tolerance. We apply NNs trained with other algorithms and our one to a character recognition problem. It is shown that our one is superior to other ones in reliability, extra learning time and/or extra learning cycles. Besides we clarify that our algorithm never degrades the generalization ability of NNs although it coerces the weights within the calculated range.},
keywords={},
doi={},
ISSN={},
month={November},}
Copy
TY - JOUR
TI - On a Weight Limit Approach for Enhancing Fault Tolerance of Feedforward Neural Networks
T2 - IEICE TRANSACTIONS on Information
SP - 1931
EP - 1939
AU - Naotake KAMIURA
AU - Teijiro ISOKAWA
AU - Yutaka HATA
AU - Nobuyuki MATSUI
AU - Kazuharu YAMATO
PY - 2000
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E83-D
IS - 11
JA - IEICE TRANSACTIONS on Information
Y1 - November 2000
AB - To enhance fault tolerance ability of the feedforward neural networks (NNs for short) implemented in hardware, we discuss the learning algorithm that converges without adding extra neurons and a large amount of extra learning time and cycles. Our algorithm modified from the standard backpropagation algorithm (SBPA for short) limits synaptic weights of neurons in range during learning phase. The upper and lower bounds of the weights are calculated according to the average and standard deviation of them. Then our algorithm reupdates any weight beyond the calculated range to the upper or lower bound. Since the above enables us to decrease the standard deviation of the weights, it is useful in enhancing fault tolerance. We apply NNs trained with other algorithms and our one to a character recognition problem. It is shown that our one is superior to other ones in reliability, extra learning time and/or extra learning cycles. Besides we clarify that our algorithm never degrades the generalization ability of NNs although it coerces the weights within the calculated range.
ER -
English
