Build1 distinct publisher3 min readPublished
Jordy Zomer built a Datalog engine so an agent maintains what it currently knows instead of searching its own transcript, and his own benchmark runs put the weak link in the model that writes the facts.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
The retraction path is where this design earns its keep. Zomer's worked example starts with three observations: an attacker controls one object, that object points to a second object, and the second object is a kernel object. A rule derives that the attacker controls a kernel object [6]. The derivation is stored, not just its result. So when later debugging disproves the pointer relationship, the engine drops the conclusions that stood on it and leaves alone any conclusion a second independent path still supports, and it can print a provenance tree for either outcome [6].
That is ordinary program-analysis discipline: record facts, derive through explicit rules, track dependencies, recompute only the affected results [5]. The interesting choice is where the boundary sits. The model still reads source code, debugger output and natural-language notes; once those become structured facts, the deterministic engine owns them [5]. Temporal facts keep the history, so a primitive can be viable across one interval and invalid after a later observation, and the log answers both what is believed now and why a dead path was explored [8]. This comes from someone doing static analysis, fuzzing and binary exploitation for a living; Ubuntu credits Zomer and Alexandra Sandulescu with two speculative-execution kernel flaws, CVE-2023-0458 and CVE-2023-0459 [17].
Now the numbers, which are claims about somebody else's workload. LongMemEval and LoCoMo are conversational-memory benchmarks, not vulnerability investigations [10]. Zomer reports 0.533 F1 on LoCoMo across three runs [14]. The score closest to his stated goal is the LongMemEval knowledge-updates category at 0.579 in a representative run [12], which sits 0.116 above the project's own overall LongMemEval F1 [3]. For any of that to predict behaviour in your agent, your workload has to be question answering over chat sessions where the relevant facts are already written as sentences. A multi-hour session in a large codebase is not that; there the facts have to be pulled out of a debugger first [3].
Which is exactly what the weak result measures. Multi-session came in at 0.211 against PropMem's 0.582, roughly 36 percent of it [13][2], and Zomer traced many of the misses to extraction failures, stating the limit plainly: a perfect rule engine cannot reason over an event the LLM never converted into a fact [13]. A perfect rule engine over an empty fact table is very fast and very quiet. The same shape shows up in the inference misses, where compressing a conditional preference into a simple tuple discards the qualifier [15]. That is schema work, not engine work, and it is where the adoption time goes.
The repository is MIT-licensed and ships an MCP server, so wiring it into Claude Code or Kimi CLI is not the cost [9]. The cost is the fact schema, the rules, and the retrieval you keep running anyway, because Lemmalog holds structured facts alongside the original source text, embeddings and lexical search [16]. Zomer's framing of the split is narrower than the pitch a vendor would make here: similarity search answers which old information looks relevant, the deductive state answers which conclusions remain supported [16]. He also concedes the extraction model can still encode a bad fact, and argues the gain is that the resulting state is inspectable and corrections have mechanical consequences once the error is found [7].
In my context that is the right trade: long analysis sessions where a revived wrong belief is expensive, and where I am willing to write the rules myself. If the job is remembering what a user said three sessions ago, the published scores point elsewhere [11].
Ranked by verification strength, evidence, and original report placement.
Zomer reports a LongMemEval F1 score of 0.463 +/- 0.010 across three runs, behind the published PropMem result of 0.550 and slightly below SimpleMem at 0.480.
Security researcher Jordy Zomer published the design and source code for Lemmalog on August 28th, per runtimewire.com, which names pwning.systems as the primary source.
Lemmalog is a Datalog engine that maintains what an AI agent currently knows, built out of a recurring failure in long-running AI-assisted vulnerability research.
Zomer's stated annoyance: after several hours inside a large codebase, an LLM agent would revive discarded exploit paths, forget that an assumption had failed, or keep reasoning from an observation invalidated in a debugger; retrieval could bring old notes back but left the model to reconcile which version remained true.
"I want it to maintain what we currently know," Zomer wrote in his August 28th technical account.
Lemmalog applies program-analysis discipline: record facts, derive conclusions through explicit rules, track their dependencies, and update only the affected results when an input changes. The model still interprets source code, debugger output and natural-language notes; once those observations become structured facts, the deterministic engine takes over.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 28, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
LoreKit puts agent memory in Markdown files you can grep, not a vendor's database1 distinct publisher
build
Agent memory products differ on one thing: whether anything decides a fact is dead1 distinct publisher
build
The one signal agent memory learns from is the one production traffic almost never sends1 distinct publisher
product
A 2x LLM bill is not a bug report: token spend is an observability problem1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One account, checkable code
Two kinds of evidence sit side by side here and they are not equal. The engine's design is verifiable by anyone: MIT-licensed code, a stated feature list, an MCP server you can point at Claude Code. The numbers are not — every score comes from Zomer's own three-run harness, and runtimewire.com says outright that none of it has been independently validated. The single fact with an outside anchor is the Ubuntu credit for CVE-2023-0458 and CVE-2023-0459, which vouches for the author rather than the engine.
Author-only footprint
Countable uptake amounts to one release and six of the author's own benchmark runs. The MCP server names Claude Code and Kimi CLI as connection targets, but nobody in this reporting is shown having wired either to Lemmalog, and there are no forks, downstream projects or production users cited. Read the figure as the floor that publication sets, not as traction.
Publishes its own losses
Unusual direction for a launch story: the framing runs cooler than the material would allow. Zomer puts out scores that trail PropMem overall and reach roughly a third of it on multi-session, names his own extraction model as the weak link, and warns that conversational benchmarks do not stand in for a vulnerability investigation. runtimewire.com keeps that posture down to the subhead. What little stretch exists is in the architectural ambition, asserted in a section where no outsider has yet tested it.
Reputation, not revenue
Nobody in this story is selling anything. MIT licence, a personal research repository, no pricing, no company — the return to Zomer is standing in a field where his name already attaches to two kernel CVEs. The pressure that does exist is positional: the report places Lemmalog against venture-funded rivals, which gives an unfunded researcher a reason to benchmark loudly, and the numbers he chose to publish cut the other way.
Precise, unreplicated
Middling, and for structural reasons rather than sloppy ones. The technical account holds together, states its numbers to three decimals with variance across runs, and identifies its own failure mode — details that are hard to fabricate and easy to test later. But there is one outlet, one primary post, and no replication of anything. Confidence at this level means plausible and specifically stated, not confirmed.