We investigate a sparsely encoded Hopfield model with unit replacement by using a statistical mechanical method called self-consistent signal-to-noise analysis. We theoretically obtain a relation between the storage capacity and the number of replacement units for each sparseness a. Moreover, we compare the unit replacement model with the forgetting model in terms of the network storage capacity. The results show that the unit replacement model has a finite value of the optimal sparseness on an open interval 0 (1/2 coding) < a < 1 (the limit of sparseness) to maximize the storage capacity for a large number of replacement units, although the forgetting model does not.

In this paper, dependence of storage capacity of an analogue associative memory model using nonmonotonic neurons on static synaptic noise and static threshold noise is shown. This dependence is analytically calculated by means of the self-consistent signal-to-noise analysis (SCSNA) proposed by Shiino and Fukai. It is known that the storage capacity of an associative memory model can be improved markedly by replacing the usual sigmoid neurons with nonmonotonic ones, and the Hopfield model has theoretically been shown to be fairly robust against introducing the static synaptic noise. In this paper, it is shown that when the monotonicity of neuron is high, the storage capacity decreases rapidly according to an increase of the static synaptic noise. It is also shown that the reduction of the storage capacity is more sensitive to an increase in the static threshold noise than to the increase in the static synaptic noise.

An associative memory model with a forgetting process a la Mezard et al. is investigated for a piecewise nonmonotonic output function by the SCSNA proposed by Shiino and Fukai. Similar to the formal monotonic two-state model analyzed by Mezard et al. , the discussed nonmonotonic model is also free from a catastrophic deterioration of memory due to overloading. We theoretically obtain a relationship between the storage capacity and the forgetting rate, and find that there is an optimal value of forgetting rate, at which the storage capacity is maximized for the given nonmonotonicity. The maximal storage capacity and capacity ratio (a ratio of the storage capacity for the conventional correlation learning rule to the maximal storage capacity) increase with nonmonotonicity, whereas the optimal forgetting rate decreases with nonmonotonicity.