Build1 publisher3 min readPublished
A hand-written memory graph matched 600 rewarded trials at 1.00 accuracy
The substrate reads keys, traces, signs and strengths and takes all four at face value, so write access to the graph is authorship of behavior, including a transplant that scored 0.17 against 0.33 chance.
The Engineer · Build desk

What happened
- Transplanting one individual's learned associations into an identical brain drove accuracy to 0.17 on a six-cue, three-action task, steering the subject toward another individual's answers.
- A function called author_memory() builds a memory graph with zero training trials, no reward and no encode() call, observing each cue's settled state and writing a trace aimed at a chosen action.
- The post sets out three moves available to anyone with write access: transplant another individual's memory, poison selected cues while the rest looks normal, or author a history that was never lived.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint A cryptographically perfect store answers only whether an unauthorized party modified a row, so the controls most teams already ship cannot distinguish an earned trace from one written this morning.
- exposure Anyone holding write credentials on an agent's external memory sits in the behavior-authoring position. That position changes who counts as a privileged actor in the threat model.
- decision The four arrays the substrate reads have no field for an origin, so adding provenance means changing the memory record itself and the read path that consumes it.
- contradiction The two halves of the threat come from separate runs: the transplant produced 0.17, while the fabricated memory was deliberately aimed at the correct answers, leaving below-chance authoring as an inference the post states but does not score.
Call `author_memory(net, env)` and the learning path never runs. For each cue the function resets the network state, steps it eight times with `net.step(u, None)`, and stores the normalized settled state `net.x` as the key [5]. The trace paired with that key is `net.W_out[env.mapping[cue]]`, the readout row for the action the attacker chose [6]. Sign and strength are filled with ones [7]. No reward flows, and `encode()` stays uncalled [4].
In this substrate that is a complete memory, because a memory is keys, traces, signs and strengths [8]. The function that converts the graph into a bias current, `observe_and_modulate()`, reads the trace without checking where it came from [9].
The fresh brain on its fixed innate policy scored 0.82 [10]. A memory earned over 600 rewarded trials took it to 1.00 [11]. The fabricated graph took it to 1.00 as well, and the post reports the substrate could not tell the two apart [12]. The gain is 0.18 either way, once after 600 trials and once after none [2].
The attacker in the demo is idealized, and the post says so. It knows the substrate's readout weights, which the author argues is exactly what write access to the graph implies [13]. "The point is not that authoring is easy for a black-box attacker," the author wrote. "The point is that provenance is never checked, so a fabricated graph is accepted and obeyed identically to a lived one" [14].
Two conditions have to hold before either number transfers to a system you operate. The stored memory has to feed behavior the way this graph does: the author describes it as "a bias current that shapes what the agent becomes before it decides anything" [17], applied before the policy decides. And the write path has to reach the same representation the policy reads out from, which in the simulation it does by construction, since the attacker is a Python function holding the network object [6].
The below-chance result and the authoring result come from different runs. The 0.17 came from transplanting one individual's learned associations into an identical brain, steering it toward another individual's answers [1], which is 0.16 under the 0.33 chance baseline [1]. The fabricated memory was aimed at the true mapping on purpose, to show the graph is obeyed, and the post says an attacker could aim the same trick at targets that drive the agent below chance [15]. The post reports no such run.
So the defensive consequence is narrow. Signing proves who wrote a row and encryption proves nobody else read it; neither answers whether the memory was ever lived [16]. A provenance check would have to live inside the memory structure, and that structure is four arrays [8].
What to watch
- Whether anyone reproduces the zero-trial result without white-box access to the substrate's readout weights.
- Whether a provenance field lands in the memory structure, and whether observe_and_modulate() is changed to consult it.
- Whether the poison variant is published: targeted traces that bias specific cues wrong while the rest of the system scores normally.