A Learning Algorithm with Activation Function Manipulation for Fault Tolerant Neural Networks
Summary :
In this paper we propose a learning algorithm to enhance the fault tolerance of feedforward neural networks (NNs for short) by manipulating the gradient of sigmoid activation function of the neuron. We assume stuck-at-0 and stuck-at-1 faults of the connection link. For the output layer, we employ the function with the relatively gentle gradient to enhance its fault tolerance. For enhancing the fault tolerance of hidden layer, we steepen the gradient of function after convergence. The experimental results for a character recognition problem show that our NN is superior in fault tolerance, learning cycles and learning time to other NNs trained with the algorithms employing fault injection, forcible weight limit and the calculation of relevance of each weight to the output error. Besides the gradient manipulation incorporated in our algorithm never spoils the generalization ability.
- Publication
- IEICE TRANSACTIONS on Information Vol.E84-D No.7 pp.899-905
- Publication Date
- 2001/07/01
- Publicized
- Online ISSN
- DOI
- Type of Manuscript
- PAPER
- Category
- Fault Tolerance
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Naotake KAMIURA, Yasuyuki TANIGUCHI, Yutaka HATA, Nobuyuki MATSUI, "A Learning Algorithm with Activation Function Manipulation for Fault Tolerant Neural Networks" in IEICE TRANSACTIONS on Information,
vol. E84-D, no. 7, pp. 899-905, July 2001, doi: .
Abstract: In this paper we propose a learning algorithm to enhance the fault tolerance of feedforward neural networks (NNs for short) by manipulating the gradient of sigmoid activation function of the neuron. We assume stuck-at-0 and stuck-at-1 faults of the connection link. For the output layer, we employ the function with the relatively gentle gradient to enhance its fault tolerance. For enhancing the fault tolerance of hidden layer, we steepen the gradient of function after convergence. The experimental results for a character recognition problem show that our NN is superior in fault tolerance, learning cycles and learning time to other NNs trained with the algorithms employing fault injection, forcible weight limit and the calculation of relevance of each weight to the output error. Besides the gradient manipulation incorporated in our algorithm never spoils the generalization ability.
URL: https://global.ieice.org/en_transactions/information/10.1587/e84-d_7_899/_p
Copy
@ARTICLE{e84-d_7_899,
author={Naotake KAMIURA, Yasuyuki TANIGUCHI, Yutaka HATA, Nobuyuki MATSUI, },
journal={IEICE TRANSACTIONS on Information},
title={A Learning Algorithm with Activation Function Manipulation for Fault Tolerant Neural Networks},
year={2001},
volume={E84-D},
number={7},
pages={899-905},
abstract={In this paper we propose a learning algorithm to enhance the fault tolerance of feedforward neural networks (NNs for short) by manipulating the gradient of sigmoid activation function of the neuron. We assume stuck-at-0 and stuck-at-1 faults of the connection link. For the output layer, we employ the function with the relatively gentle gradient to enhance its fault tolerance. For enhancing the fault tolerance of hidden layer, we steepen the gradient of function after convergence. The experimental results for a character recognition problem show that our NN is superior in fault tolerance, learning cycles and learning time to other NNs trained with the algorithms employing fault injection, forcible weight limit and the calculation of relevance of each weight to the output error. Besides the gradient manipulation incorporated in our algorithm never spoils the generalization ability.},
keywords={},
doi={},
ISSN={},
month={July},}
Copy
TY - JOUR
TI - A Learning Algorithm with Activation Function Manipulation for Fault Tolerant Neural Networks
T2 - IEICE TRANSACTIONS on Information
SP - 899
EP - 905
AU - Naotake KAMIURA
AU - Yasuyuki TANIGUCHI
AU - Yutaka HATA
AU - Nobuyuki MATSUI
PY - 2001
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E84-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2001
AB - In this paper we propose a learning algorithm to enhance the fault tolerance of feedforward neural networks (NNs for short) by manipulating the gradient of sigmoid activation function of the neuron. We assume stuck-at-0 and stuck-at-1 faults of the connection link. For the output layer, we employ the function with the relatively gentle gradient to enhance its fault tolerance. For enhancing the fault tolerance of hidden layer, we steepen the gradient of function after convergence. The experimental results for a character recognition problem show that our NN is superior in fault tolerance, learning cycles and learning time to other NNs trained with the algorithms employing fault injection, forcible weight limit and the calculation of relevance of each weight to the output error. Besides the gradient manipulation incorporated in our algorithm never spoils the generalization ability.
ER -
English
