The Hopfield Network

What you're seeing

A grid of 576 tiny units (24×24), each holding just one bit — on (bright) or off (dark). Every unit is wired to every other, and a handful of whole-grid patterns — a heart, a ring, a checkerboard, stripes, a pinwheel — have been stored in those wires. Hand the network a damaged version of one of them, with a quarter of its pixels randomly flipped, and let it run. Unit by unit it corrects itself, the static dissolves, and a recognizable memory condenses out of the noise. No single unit "knows" the heart; the memory lives only in the pattern of connections, and recall is the network physically relaxing into it.

The rule

Two ingredients — one to store, one to recall.

• Storage (Hebb's rule). "Units that were on together in a stored pattern wire together." Each stored pattern ξ^p is a vector of ±1 over all units, and the symmetric weight between units i and j is the average agreement across the stored patterns:  W_ij = (1/N) Σ_p ξ^p_i ξ^p_j,  with no self-connection (W_ii = 0). The weights are set once and never change during recall.
• Recall (asynchronous update). Pick a unit and set it to agree with the weighted vote of all the others:  s_i ← sign( Σ_j W_ij s_j ). Repeat, one unit at a time, in random order. (This sim never builds the full N×N matrix — it computes the same vote directly from the stored patterns, which is faster and exactly equivalent.)

The magic is that every such update can only ever lower a single global quantity, the energy  E = −½ Σ_ij W_ij s_i s_j. So the state can never climb; it slides monotonically downhill and comes to rest at the bottom of an energy valley — a fixed point where no unit wants to flip. Hebbian storage carves a valley around each stored pattern, so the network settles into whichever memory the cue was nearest. Watch the energy readout fall and then flatten: that flattening is the moment of recall.

Why it matters

This is content-addressable memory and collective computation emerging from dumb two-state parts. Ordinary computer memory is addressed by location — you must know where a thing is stored to retrieve it. A Hopfield network is addressed by content: hand it a fragment or a noisy version, and it completes and corrects it. No unit holds a memory; the memories live in the pattern of connections, and recall is a physical relaxation to an energy minimum, exactly like a ball rolling into the nearest bowl. John Hopfield's 1982 paper that introduced this is literally titled "Neural networks and physical systems with emergent collective computational abilities" — the emergence is the whole point. It made the energy-landscape picture of memory concrete, seeded the statistical physics of neural networks, and shares its mathematics with spin glasses and with this Atlas's Ising model (the same energy E = −½ Σ J_ij s_i s_j over ±1 spins). The work was recognized with the 2024 Nobel Prize in Physics, shared by Hopfield and Geoffrey Hinton.

There is a hard limit, and it is instructive. A network of N units can store only about 0.138·N random patterns before recall breaks down (Amit, Gutfreund & Sompolinsky, 1985). Push the stored slider past that capacity and the valleys start to merge: clean recall decays into spurious states — blends and mixtures of several memories that the network invents but was never taught. That failure mode is not a bug in the demo; it is a real and well-studied property of the model.

In the wild

The Hopfield network is the textbook model of associative and content-addressable memory, and a clean illustration of error correction by relaxation: a corrupted signal flows back to the nearest stored "codeword." More broadly it gave us the energy-landscape view of memory and computation — the idea that solving a problem can mean letting a physical system roll downhill to a minimum, which also underlies optimization networks and Boltzmann machines. The idea is alive in current research: "modern Hopfield networks" with a sharper energy function can store exponentially many patterns and turn out to be mathematically equivalent to the attention mechanism at the heart of Transformer language models.

Honesty check: this is a principle-capturing toy, not a model of a brain. Real neurons are not binary, their connections are not symmetric, and no brain region is fully connected with weights frozen by a single Hebbian snapshot. The value here is the clean demonstration that robust, content-addressable memory and a tiny analog computer can emerge from simple, identical units wired by one local rule — not that biological memory works this way in detail. It does not.

Try this

• Recall a pattern at low corruption (drag the slider down toward 10%) and watch the network snap to perfect recovery in a sweep or two — overlap races to 1.0.
• Crank corruption toward 50% and run it several times. Now the cue is barely related to the original; recall sometimes fails or lands on the wrong memory, because the noisy state fell into a neighbouring valley.
• Raise stored toward 6 of 6 to crowd more memories into the same weights, then recall a pattern from a high-corruption cue. With more stored patterns the valleys interfere, so recovery gets harder and can stall in a spurious blend below overlap 1.0. (The demo's six patterns are chosen to be fairly distinct, so they stay near capacity — push the corruption up to coax the interference into view.)
• Press noise for a totally random start, or draw your own pattern by dragging on the grid, then run it — the network will collapse your scribble into whichever stored memory it most resembles.

The other Networks entries — Preferential Attachment and Slime Mould Networks — are about how a physical or transport network grows. This one is the complementary facet: how dynamics running on a fixed network can compute and remember.

References

1. Hopfield, J. J. (1982). "Neural networks and physical systems with emergent collective computational abilities." Proceedings of the National Academy of Sciences 79(8), 2554–2558.
2. Hopfield, J. J. (1984). "Neurons with graded response have collective computational properties like those of two-state neurons." Proceedings of the National Academy of Sciences 81(10), 3088–3092.
3. Amit, D. J., Gutfreund, H. & Sompolinsky, H. (1985). "Storing infinite numbers of patterns in a spin-glass model of neural networks." Physical Review Letters 55(14), 1530–1533. (Capacity ≈ 0.138·N.)
4. Hebb, D. O. (1949). The Organization of Behavior. Wiley. (The "fire together, wire together" rule.)
5. "The Nobel Prize in Physics 2024" (John J. Hopfield and Geoffrey E. Hinton). nobelprize.org.