Four substitutions turn a Hopfield update into scaled dot-product attention

Read the update rule as a lookup. Compare the current state with every stored pattern, turn the similarities into softmax weights, return the weighted sum of the patterns ^[5]. Then substitute: stored patterns become keys, the state becomes a query, a value projection is added, and beta is set to 1/sqrt(d_k) ^[6]. What falls out is softmax(QK^T/sqrt(d_k))V ^[6]. At the level of that one operation the equality is exact ^[7]. beta is pinned by the head width, so a head of width 64 runs at 0.125 ^[23].

It is one update step. The paper's own text says "retrieval with one update is compatible with activating the layers of deep networks" ^[8]. Its theorems bound how close that single update gets to the stored pattern when the patterns are well separated ^[9]. Residual connections, layer normalisation, the feed-forward blocks, causal masking and training dynamics are outside it ^[12].

The identity is also a coincidence of form ^[2]. Attention Is All You Need appeared in 2017, three years before the equivalence was shown, and the two were found to be the same operation after the fact ^[10]. The first figure of the Hopfield paper uses an equality sign, and its caption explains that the sign means "keeps the properties" ^[11].

In the thread the dev.to post is arguing with, "attention is just a Hopfield network" becomes "so the model already has associative memory, so an agent does not need anything else to remember" ^[1]. The post's answer is that the equality does not turn a context window into memory that outlives the conversation ^[14], and that the loose reading is where architecture decisions go wrong ^[15]. It does not name a team that made that call. For the theorem to license skipping persistent storage, one-step retrieval over patterns already in the window would have to be what an agent needs between sessions. The result covers one step. The network iterating to convergence sits outside the result ^[26].

Capacity is where the numbers stop transferring. Three figures get quoted as if they answered the same question: an empirical rule of thumb from small simulations, a statistical mechanics result that allows a little noise in what comes back, and a bound that demands exact recovery and so is smaller and grows more slowly ^[16]. Dense associative memories raised capacity to polynomial and then exponential scaling, and Ramsauer and colleagues prove exponential capacity in the dimension of the space for continuous states ^[17]. Quote 0.14N as the capacity of the network inside a transformer and you are quoting the 1987 model ^[18].

The most practical part of the paper, by the post's reckoning, is the taxonomy of fixed points for the update. A query can settle on a single stored pattern when one is close and well separated, on a metastable state that averages a subset of similar patterns, or on a global average of everything stored when nothing is well separated ^[19]. The authors then used that lens on trained models ^[25]. The abstract reports that attention heads "perform in the first layers preferably global averaging and in higher layers partial averaging via metastable states" ^[20]. A head returning a blend may be sitting at a metastable fixed point.

The title nods at the 2017 paper, and the genre is large enough to have been counted. A December 2025 preprint, All You Need is Not All You Need for a Paper Title, finds 717 arXiv titles containing the phrase between 2009 and 2025, 200 of them in 2025 alone, and argues the format favours "memorability over precision" ^[21]. That is 28 percent of seventeen years in a single year ^[22]. Here the title states the theorem ^[24]. Later work narrows the conditions under which the identity holds, and the identity stands ^[13].