Capacity of the Hebbian-Hopfield network associative memory

In citeHop82, Hopfield introduced a Hebbian learning rule based neural network model and suggested how it can efficiently operate as an associative memory. Studying random binary patterns, he also uncovered that, if a small fraction of errors is tolerated in the stored patterns retrieval, the capacity of the network (maximal number of memorized patterns, m) scales linearly with each pattern's size, n. Moreover, he famously predicted αc=limnrightarrow∞fracmn≈ 0.14. We study this very same scenario with two famous pattern's basins of attraction: textbf(i) The AGS one from citeAmiGutSom85; and textbf(ii) The NLT one from citeNewman88,Louk94,Louk94a,Louk97,Tal98. Relying on the fully lifted random duality theory (fl RDT) from citeStojnicflrdt23, we obtain the following explicit capacity characterizations on the first level of lifting: beginequation αc(AGS,1) = left ( max_δ∈ left ( 0,frac12right ) frac1-2δ√2 mboxerfinv left ( 1-2δright ) - frac2√2π e^-left ( mboxerfinvleft ( 1-2δ right )right )2right )2 ≈ 0.137906 endequation beginequation αc(NLT,1) = fracmboxerf(x)22x2-1+mboxerf(x)2 ≈ 0.129490, quad 1-mboxerf(x)2- frac2mboxerf(x)e-x2√πx+frac2e-2x2π=0. endequation A substantial numerical work gives on the second level of lifting αc(AGS,2) ≈ 0.138186 and αc(NLT,2) ≈ 0.12979, effectively uncovering a remarkably fast lifting convergence. Moreover, the obtained AGS characterizations exactly match the replica symmetry based ones of citeAmiGutSom85 and the corresponding symmetry breaking ones of citeSteKuh94.