In the present paper, we attempt to show that the information about input patterns must be as small as possible for improving the generalization performance under the condition that the network can produce targets with appropriate accuracy. The information is defined with respect to the hidden unit activity and we suppose that the hidden unit has a crucial role to store the information content about input patterns. The information is defined by the difference between uncertainty of the hidden unit at the initial stage of the learning and the uncertainty of the hidden unit at the final stage of the learning. After having formulated an update rule for the information minimization, we applied the method to a problem of language acquisition: the inference of the past tense forms of regular and irregular verbs. Experimental results confirmed that by our method, the information was significantly decreased and the generalization performance was greatly improved.

In this paper, we propose a method, called Kernel Hidden Unit Analysis, to reduce the network size. The kernel hidden unit analysis in composed of two principal components: T-component and S-component. The T-component transforms original networks into the networks which can easily be simplified. The S-component is used to select kernel units in the networks and construct kernel networks with kernel units. For the T-component, an entropy function is used, which is defined with respect to the state of the hidden units. In a process of entropy minimization, multiple strongly inhibitory connections are to be generated, which tend to turn off as many units as possible. Thus, some major hidden units can easily be extracted. Concerning the S-component, we use the relevance and the variance of input-hidden connections and detect the kernel hidden units for constructing the kernel network. Applying the kernel hidden unit analysis to the symmetry problem and autoencoders, we perfectly succeeded in obtaining kernel networks with small entropy, that is, small number of hidden units.

Certificate Revocation is a critical issue for a practical, public-key infrastructure. A new efficient revocation protocol using a one-way hash tree structure (instead of the classical list structure, which is known as a standard for revocation), was proposed and examined to reduce communication and computation costs. In this paper, we analysis a k-ary hash tree for certificate revocation and prove that k = 2 minimizes communication cost.