Skip to content

Build1 publisher3 min readPublished

Cosine similarity ranks the menu above the peanut allergy note

Statewave's case for a separate agent memory layer rests on three defaults in a RAG stack: similarity-only ranking, append-only chunks, and no compaction. The post asserts all three failure modes and measures none of them.

The Engineer · Build desk

Illustration accompanying Cosine similarity ranks the menu above the peanut allergy note

What happened

  • A Statewave post on dev.to argues that RAG and agent memory share an embedding store, usually pgvector, and both use approximate-nearest-neighbour search at retrieval time, and that the overlap ends there.
  • RAG, in the post's account, stores chunks of markdown pages, PDFs and support articles, and returns the chunks most semantically similar to the question for splicing into the prompt as grounding.
  • Memory instead stores typed records: episodes covering raw turns, tool calls and decisions, plus compiled facts derived from them, ranked on recency, kind, validity and similarity.
  • The post names three failure modes that show up when teams stretch RAG to do memory's job: similarity that is not decision-relevant, no compaction, and no invalidation model.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • decision Add compaction, kind priority and validity windows to your application code and you are running a memory runtime under a different name, so the real question is who maintains the compiler.
  • cost Compilation is an extra pass over raw episodes, so each turn costs a write, a compile and a ranked read, and whoever operates the Postgres instance pays for all three.
  • capability Same query, same bundle makes a bad prompt reproducible, and a memory bug you can bisect is a different class of problem from a retriever that quietly reorders as the corpus grows.
  • precedent If the split becomes the default agent shape, teams carry two indexes and two write paths, and pressure lands on RAG frameworks to support typed records they were not designed to hold.

The allergy example carries the whole argument, and it is a claim about embedding geometry with no measurement attached. The post says the message "I'm allergic to peanuts" and the later question "What should I order for lunch?" do not have close embeddings, and that cosine similarity will surface every restaurant chunk before the allergy note [6]. For that to happen in your stack, the note has to be sitting in the same index as the menu content as an untyped chunk, and your retriever has to rank on similarity alone. Give the note a kind field and the ordering problem becomes a filter.

The compaction point is arithmetic, which makes it harder to talk your way around. The post's example compiles 200 conversation turns into one line, "user is a senior engineer at a fintech, prefers terse responses", and gives the finite prompt budget as the reason [8]. That is 200 records down to one [9]. RAG, by the post's account, indexes the corpus as it stands [8].

Invalidation is the piece I would not bolt on at query time. Deciding which of two contradictory chunks is current needs a field that somebody wrote at ingest. The post's case is that when a user's job changes, the old record has to be marked superseded instead of being returned alongside the new one, and an append-only chunk store has no notion of validity windows or supersession [10].

The remedy list does not map one-to-one onto the failure list. Compilation answers compaction, and deterministic ranking, mixing similarity with kind priority, recency, temporal validity and an explicit token budget, answers the relevance complaint [12]. Provenance is the third addition: every compiled memory carries the IDs of the episodes it came from, so an answer is auditable back to the raw event [13]. Auditability sits outside the three failure modes the post names [19]. Invalidation is handled inside compilation, as validity windows on the typed records [11].

This is Statewave's own post, and the sales line is on the page: the common architecture mistake is "the one we built Statewave to stop people from making" [18]. The engineering claim underneath holds independently of the sale. "You can paper over each of these in your application code. Teams that do end up with a memory runtime in everything but name," the post says [14].

Adoption cost, as described: pgvector underneath with no separate vector database to operate, and no document loader, chunker or retriever interface for grounding over a corpus [16], reached through what the post calls a five-minute Docker Compose path [17]. The description skips every number, including latency, price, and any accuracy comparison against the hand-rolled version [20]. There is no benchmark table here.

The recommendation is both layers, with the grounding corpus in RAG and the user, account and project context in memory [15]. Nothing in the three complaints says vector search is the wrong index for either one. What they say is that the RAG write path is wrong for facts about a person: the chunks are untyped, appends never supersede anything, and retrieval ignores recency [5].

What to watch

  • Whether Statewave publishes its ranking-signal weights or a measured comparison against a similarity-only retriever.
  • Whether a pgvector-backed memory store holds up as episode volume outgrows one Postgres instance; the post gives no scaling figures.
  • If RAG frameworks add typed chunks and validity windows to their stores, the separate memory service loses the argument it is sold on.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories