The Capacity of Sparsely Encoded Associative Memories

Mehdi N. SHIRAZI

The Capacity of Sparsely Encoded Associative Memories

Mehdi N. SHIRAZI

Full Text Views

0

Cite this

Summary :

We consider an asymptotically sparsely encoded associative memory. Patterns are encoded by n-dimensional vectors of 1 and 1 generated randomly by a sequence of biased Bernoulli trials and stored in the network according to Hebbian rule. Using a heuristic argument we derive the following capacities:c(n)n^e/4k log n'C(n)n^e/4k(1e)log n'where, 0e1 controls the degree of sparsity of the encoding scheme and k is a constant. Here c(n) is the capacity of the network such that any stored pattern is a fixed point with high probability, whereas C(n) is the capacity of the network such that all stored patterns are fixed points with high probability. The main contribution of this technical paper is a theoretical verification of the above results using the Poisson limit theorems of exchangeable events.

Publication: IEICE TRANSACTIONS on Information Vol.E76-D No.3 pp.360-367

Publication Date: 1993/03/25

Publicized

Online ISSN

DOI

Type of Manuscript: PAPER

Category: Bio-Cybernetics

Cite this

Copy

Mehdi N. SHIRAZI, "The Capacity of Sparsely Encoded Associative Memories" in IEICE TRANSACTIONS on Information, vol. E76-D, no. 3, pp. 360-367, March 1993, doi: .
Abstract: We consider an asymptotically sparsely encoded associative memory. Patterns are encoded by n-dimensional vectors of 1 and 1 generated randomly by a sequence of biased Bernoulli trials and stored in the network according to Hebbian rule. Using a heuristic argument we derive the following capacities:c(n)n^e/4k log n'C(n)n^e/4k(1e)log n'where, 0e1 controls the degree of sparsity of the encoding scheme and k is a constant. Here c(n) is the capacity of the network such that any stored pattern is a fixed point with high probability, whereas C(n) is the capacity of the network such that all stored patterns are fixed points with high probability. The main contribution of this technical paper is a theoretical verification of the above results using the Poisson limit theorems of exchangeable events.
URL: https://global.ieice.org/en_transactions/information/10.1587/e76-d_3_360/_p

Copy

@ARTICLE{e76-d_3_360,
author={Mehdi N. SHIRAZI, },
journal={IEICE TRANSACTIONS on Information},
title={The Capacity of Sparsely Encoded Associative Memories},
year={1993},
volume={E76-D},
number={3},
pages={360-367},
abstract={We consider an asymptotically sparsely encoded associative memory. Patterns are encoded by n-dimensional vectors of 1 and 1 generated randomly by a sequence of biased Bernoulli trials and stored in the network according to Hebbian rule. Using a heuristic argument we derive the following capacities:c(n)n^e/4k log n'C(n)n^e/4k(1e)log n'where, 0e1 controls the degree of sparsity of the encoding scheme and k is a constant. Here c(n) is the capacity of the network such that any stored pattern is a fixed point with high probability, whereas C(n) is the capacity of the network such that all stored patterns are fixed points with high probability. The main contribution of this technical paper is a theoretical verification of the above results using the Poisson limit theorems of exchangeable events.},
keywords={},
doi={},
ISSN={},
month={March},}

Copy

TY - JOUR
TI - The Capacity of Sparsely Encoded Associative Memories
T2 - IEICE TRANSACTIONS on Information
SP - 360
EP - 367
AU - Mehdi N. SHIRAZI
PY - 1993
DO -
JO - IEICE TRANSACTIONS on Information
SN -
VL - E76-D
IS - 3
JA - IEICE TRANSACTIONS on Information
Y1 - March 1993
AB - We consider an asymptotically sparsely encoded associative memory. Patterns are encoded by n-dimensional vectors of 1 and 1 generated randomly by a sequence of biased Bernoulli trials and stored in the network according to Hebbian rule. Using a heuristic argument we derive the following capacities:c(n)n^e/4k log n'C(n)n^e/4k(1e)log n'where, 0e1 controls the degree of sparsity of the encoding scheme and k is a constant. Here c(n) is the capacity of the network such that any stored pattern is a fixed point with high probability, whereas C(n) is the capacity of the network such that all stored patterns are fixed points with high probability. The main contribution of this technical paper is a theoretical verification of the above results using the Poisson limit theorems of exchangeable events.
ER -