Neural Network Pruning by Fusing Hidden Layer Units

Keisuke KAMEYAMA; Yukio KOSUGI

Neural Network Pruning by Fusing Hidden Layer Units

Keisuke KAMEYAMA, Yukio KOSUGI

Full Text Views

0

Cite this

Summary :

Neural network pruning is a technique to obtain a fully functional subset of a redundant network for the efficiency of computation. A new method to prune a redundant three-layered neural network by means of neural element fusion" is introduced. In contrast to conventional pruning techniques that remove unimportant portions of the network, our method fuses a pair of hidden layer units so that features accumulated in both units are preserved as possible. The pair of hidden layer units to be fused is chosen by evaluating a firing similarity. This similarity measure also informs when the pruning should be stopped. The fusing method was compared with well known unit removing" methods on computer simulations. The results show that our fusing method considerably reduces the error increase due to the pruning, even in subminimal networks where conventional methods are ineffective. This enables to cut down the total cost of computation to reach the minmal network configuration.

Publication: IEICE TRANSACTIONS on Information Vol.E74-D No.12 pp.4198-4204

Publication Date: 1991/12/25

Publicized

Online ISSN

DOI

Type of Manuscript: LETTER

Category: Bio-Cybernetics

Authors

Keisuke KAMEYAMA
Yukio KOSUGI

Keyword

Cite this

Copy

Keisuke KAMEYAMA, Yukio KOSUGI, "Neural Network Pruning by Fusing Hidden Layer Units" in IEICE TRANSACTIONS on Information, vol. E74-D, no. 12, pp. 4198-4204, December 1991, doi: .
Abstract: Neural network pruning is a technique to obtain a fully functional subset of a redundant network for the efficiency of computation. A new method to prune a redundant three-layered neural network by means of neural element fusion" is introduced. In contrast to conventional pruning techniques that remove unimportant portions of the network, our method fuses a pair of hidden layer units so that features accumulated in both units are preserved as possible. The pair of hidden layer units to be fused is chosen by evaluating a firing similarity. This similarity measure also informs when the pruning should be stopped. The fusing method was compared with well known unit removing" methods on computer simulations. The results show that our fusing method considerably reduces the error increase due to the pruning, even in subminimal networks where conventional methods are ineffective. This enables to cut down the total cost of computation to reach the minmal network configuration.
URL: https://global.ieice.org/en_transactions/information/10.1587/e74-d_12_4198/_p

Copy

@ARTICLE{e74-d_12_4198,
author={Keisuke KAMEYAMA, Yukio KOSUGI, },
journal={IEICE TRANSACTIONS on Information},
title={Neural Network Pruning by Fusing Hidden Layer Units},
year={1991},
volume={E74-D},
number={12},
pages={4198-4204},
abstract={Neural network pruning is a technique to obtain a fully functional subset of a redundant network for the efficiency of computation. A new method to prune a redundant three-layered neural network by means of neural element fusion" is introduced. In contrast to conventional pruning techniques that remove unimportant portions of the network, our method fuses a pair of hidden layer units so that features accumulated in both units are preserved as possible. The pair of hidden layer units to be fused is chosen by evaluating a firing similarity. This similarity measure also informs when the pruning should be stopped. The fusing method was compared with well known unit removing" methods on computer simulations. The results show that our fusing method considerably reduces the error increase due to the pruning, even in subminimal networks where conventional methods are ineffective. This enables to cut down the total cost of computation to reach the minmal network configuration.},
keywords={},
doi={},
ISSN={},
month={December},}

Copy

TY - JOUR
TI - Neural Network Pruning by Fusing Hidden Layer Units
T2 - IEICE TRANSACTIONS on Information
SP - 4198
EP - 4204
AU - Keisuke KAMEYAMA
AU - Yukio KOSUGI
PY - 1991
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E74-D
IS - 12
JA - IEICE TRANSACTIONS on Information
Y1 - December 1991
AB - Neural network pruning is a technique to obtain a fully functional subset of a redundant network for the efficiency of computation. A new method to prune a redundant three-layered neural network by means of neural element fusion" is introduced. In contrast to conventional pruning techniques that remove unimportant portions of the network, our method fuses a pair of hidden layer units so that features accumulated in both units are preserved as possible. The pair of hidden layer units to be fused is chosen by evaluating a firing similarity. This similarity measure also informs when the pruning should be stopped. The fusing method was compared with well known unit removing" methods on computer simulations. The results show that our fusing method considerably reduces the error increase due to the pruning, even in subminimal networks where conventional methods are ineffective. This enables to cut down the total cost of computation to reach the minmal network configuration.
ER -