build1 publisher
Four substitutions turn a Hopfield update into scaled dot-product attention
Ramsauer and colleagues proved the equality in 2020, three years after attention was published. It holds for a single retrieval step inside a forward pass, and the capacity figures quoted alongside it come from a different model.
Publishers:dev.to
Reality
- Evidence74
- Adoption
- Insufficient
- Hype gap−8
- Incentives62
- Confidence66