NoteBank lets Gated DeltaNet re-read its own saved snippets and reach 70.3% at 1M tokens
KAUST's NoteBank lifts Gated DeltaNet's mean accuracy on six 1M-token RULER tasks from 1.7% to 70.3% with no added training. The paper's fix for state forgetting is a small archive of verbatim snippets replayed before each question.
The Engineer · Build desk
Key figures: NoteBank's persistent storage is 74.5 MiB, against 192 GiB for a full multi-head-attention KV cache at matched width, depth and precision, per the paper's Figure 1. Its extra compute is reported as at most 0.025% of analytical FLOPs.
| Figure | Value | Note | Claim |
|---|---|---|---|
| NoteBank persistent storage, paper's Figure 1 | 74.5 MiB | 13 | |
| Full multi-head-attention KV cache | 192 GiB | matched width, depth and precision | 13 |
| Extra compute, at most | 0.025% | of analytical FLOPs, as reported | 14 |
What happened
- NoteBank leaves the backbone's weights and architecture unchanged and uses no external embedding model, trained retriever or separate reader network.
- At write time it scores each token by how far its key vector sits from the chunk's mean key, then stores the most distinctive windows as notes.
- At question time, each layer-head channel votes for its best-matching note, and the top-voted notes, up to R of them, are retrieved.
- The paper reports persistent storage of 74.5 MiB, against 192 GiB for a full multi-head-attention KV cache of the same width, depth and precision, or about 1/2,600 of the size.
- The paper puts the extra compute at no more than 0.025% of analytical FLOPs.
Why it matters
- What it limits constraint Replay holds at most R notes, so a question whose evidence is spread over more than R passages cannot be replayed in full.
- What it costs cost Archive size and the exhaustive match against it grow linearly with context length while replay cost stays flat, so the scan is the term to measure at 1M tokens.
- What's now possible capability A team running Gated DeltaNet or Mamba-2 gets a recall path with no separate retriever or embedding model to host.
The numbers below are the authors' own, relayed by a dev.to write-up of the KAUST paper [1]. The write-up does not report a test by anyone outside the group.
On the six needle tasks, Gated DeltaNet gains 68.6 points [11]. A mean of 70.3% still leaves it 29.7 points short of a perfect score [12]. The paper reports no significant retrieval degradation from 4K to 1M tokens [8].
The write step is where we would test first. It selects notes before any question is known, so it can only use a generic signal [4]. For the 70.3% to transfer, the facts your workload asks about would have to look distinctive against the text around them. If an answer is phrased like its neighbours, we'd expect its key vector to sit near the chunk mean and its window to lose out to higher-scoring ones within the token budget [3].
The read side is simpler. The retrieved notes go in document order through the unchanged backbone, starting from a fresh copy of the saved state, and the question comes after them [6]. The model processes the evidence immediately before it has to use it.
In our view, reusing the model's own key and query vectors is the right design choice. It keeps the deployment surface to the backbone you already run [2]. The caveat is the phrase "training-free". It covers the headline retrieval results. The paper also trains lightweight LoRA adapters to help the model use replayed evidence on downstream tasks, and they are enabled only during replay and answering [9].
What to watch
- Results on tasks beyond the six RULER needle tasks, reported with and without the LoRA adapters.
- Independent runs on corpora where the needed fact does not look distinctive against its surrounding text.
- Wall-clock latency of the exhaustive archive scan at 1M tokens.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence40
- Adoption
- Insufficient
- Hype gap+10
- Incentives
- Insufficient
- Confidence55
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
A KAUST paper, "History Replays Itself" (arXiv:2610.15715), proposes NoteBank, a training-free add-on that stores a few raw text snippets on the side and replays them through the model when a question arrives.
- [2]
NoteBank does not change the backbone's weights or architecture, and does not use an external embedding model, a separately trained retriever, or a separate reader network.
- [3]
Within each chunk, the writer scores every token by one minus the cosine similarity between the token's key and the chunk's mean key, averaged over layers and heads. Windows with high total distinctiveness are chosen, expanded to sentence boundaries within a token budget, and stored as notes.
- [4]
Note selection happens before any question is known, which is why selection has to rely on a generic signal like distinctiveness.
- [5]
When a question arrives, it is run from a copy of the saved state to get its own query vectors. Each layer-head channel scores every note by maximum cosine similarity and votes for its best match, and the notes with the most votes, up to R of them, are retrieved. Ties go to earlier notes.
- [6]
The retrieved notes are concatenated in original document order and fed through the unchanged backbone, starting from a fresh copy of the saved state. Then the question is processed and the answer generated.
- [7]
On six RULER needle-in-a-haystack tasks at one million tokens, the paper reports Gated DeltaNet's mean accuracy going from 1.7% to 70.3% with NoteBank.
- [8]
The paper reports no significant retrieval degradation from 4K to 1M tokens.
- [9]
The paper optionally trains lightweight LoRA adapters to help the model use replayed evidence on downstream tasks. The headline retrieval results are training-free, and the adapters are only enabled during replay and answering.
- [10]
Archive size grows linearly with context length at a rate set by the note budgets. Matching a question against the archive is exhaustive, so it also grows linearly. The replay budget is constant, at most R times the note length in extra tokens.
- [11]
Gated DeltaNet's gain on the six RULER tasks is 68.6 percentage points.
- [12]
A mean accuracy of 70.3% is 29.7 percentage points below a perfect score.
- [13]
Persistent storage is described as roughly 1/2,600 of a full multi-head-attention KV cache at matched width, depth and precision (74.5 MiB vs. 192 GiB in the paper's Figure 1).
- [14]
The extra compute is reported as at most 0.025% of analytical FLOPs.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toNoteBank: How Replaying a Few Saved Snippets Restores Long-Context Recall in Linear-Attention Models
1 article · October 15, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- Retrieval-Augmented GenerationFollow
- Linear attention and state-space modelsFollow
- Long-context memoryFollow