IEICE TRANSACTIONS on Information
Kernel Hidden Unit Analysis--Network Size Reduction by Entropy Minimization--
Summary :
In this paper, we propose a method, called Kernel Hidden Unit Analysis, to reduce the network size. The kernel hidden unit analysis in composed of two principal components: T-component and S-component. The T-component transforms original networks into the networks which can easily be simplified. The S-component is used to select kernel units in the networks and construct kernel networks with kernel units. For the T-component, an entropy function is used, which is defined with respect to the state of the hidden units. In a process of entropy minimization, multiple strongly inhibitory connections are to be generated, which tend to turn off as many units as possible. Thus, some major hidden units can easily be extracted. Concerning the S-component, we use the relevance and the variance of input-hidden connections and detect the kernel hidden units for constructing the kernel network. Applying the kernel hidden unit analysis to the symmetry problem and autoencoders, we perfectly succeeded in obtaining kernel networks with small entropy, that is, small number of hidden units.
- Publication
- IEICE TRANSACTIONS on Information Vol.E78-D No.4 pp.484-489
- Publication Date
- 1995/04/25
- Publicized
- Online ISSN
- DOI
- Type of Manuscript
- PAPER
- Category
- Bio-Cybernetics and Neurocomputing
Authors
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Ryotaro KAMIMURA, Shohachiro NAKANISHI, "Kernel Hidden Unit Analysis--Network Size Reduction by Entropy Minimization--" in IEICE TRANSACTIONS on Information,
vol. E78-D, no. 4, pp. 484-489, April 1995, doi: .
Abstract: In this paper, we propose a method, called Kernel Hidden Unit Analysis, to reduce the network size. The kernel hidden unit analysis in composed of two principal components: T-component and S-component. The T-component transforms original networks into the networks which can easily be simplified. The S-component is used to select kernel units in the networks and construct kernel networks with kernel units. For the T-component, an entropy function is used, which is defined with respect to the state of the hidden units. In a process of entropy minimization, multiple strongly inhibitory connections are to be generated, which tend to turn off as many units as possible. Thus, some major hidden units can easily be extracted. Concerning the S-component, we use the relevance and the variance of input-hidden connections and detect the kernel hidden units for constructing the kernel network. Applying the kernel hidden unit analysis to the symmetry problem and autoencoders, we perfectly succeeded in obtaining kernel networks with small entropy, that is, small number of hidden units.
URL: https://global.ieice.org/en_transactions/information/10.1587/e78-d_4_484/_p
Copy
@ARTICLE{e78-d_4_484,
author={Ryotaro KAMIMURA, Shohachiro NAKANISHI, },
journal={IEICE TRANSACTIONS on Information},
title={Kernel Hidden Unit Analysis--Network Size Reduction by Entropy Minimization--},
year={1995},
volume={E78-D},
number={4},
pages={484-489},
abstract={In this paper, we propose a method, called Kernel Hidden Unit Analysis, to reduce the network size. The kernel hidden unit analysis in composed of two principal components: T-component and S-component. The T-component transforms original networks into the networks which can easily be simplified. The S-component is used to select kernel units in the networks and construct kernel networks with kernel units. For the T-component, an entropy function is used, which is defined with respect to the state of the hidden units. In a process of entropy minimization, multiple strongly inhibitory connections are to be generated, which tend to turn off as many units as possible. Thus, some major hidden units can easily be extracted. Concerning the S-component, we use the relevance and the variance of input-hidden connections and detect the kernel hidden units for constructing the kernel network. Applying the kernel hidden unit analysis to the symmetry problem and autoencoders, we perfectly succeeded in obtaining kernel networks with small entropy, that is, small number of hidden units.},
keywords={},
doi={},
ISSN={},
month={April},}
Copy
TY - JOUR
TI - Kernel Hidden Unit Analysis--Network Size Reduction by Entropy Minimization--
T2 - IEICE TRANSACTIONS on Information
SP - 484
EP - 489
AU - Ryotaro KAMIMURA
AU - Shohachiro NAKANISHI
PY - 1995
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E78-D
IS - 4
JA - IEICE TRANSACTIONS on Information
Y1 - April 1995
AB - In this paper, we propose a method, called Kernel Hidden Unit Analysis, to reduce the network size. The kernel hidden unit analysis in composed of two principal components: T-component and S-component. The T-component transforms original networks into the networks which can easily be simplified. The S-component is used to select kernel units in the networks and construct kernel networks with kernel units. For the T-component, an entropy function is used, which is defined with respect to the state of the hidden units. In a process of entropy minimization, multiple strongly inhibitory connections are to be generated, which tend to turn off as many units as possible. Thus, some major hidden units can easily be extracted. Concerning the S-component, we use the relevance and the variance of input-hidden connections and detect the kernel hidden units for constructing the kernel network. Applying the kernel hidden unit analysis to the symmetry problem and autoencoders, we perfectly succeeded in obtaining kernel networks with small entropy, that is, small number of hidden units.
ER -
English
