Memora grades memory agents on what they should have forgotten, and six of them fail

A retrieval score cannot tell you which of two contradictory memories an agent actually used. If a user says one thing in March and the opposite in June, a system that stores both and pulls both back is credited with recall, and there is no column for which version it then acted on. FAMA is built to charge for exactly that: it penalises reliance on obsolete or invalidated memory instead of rewarding the presence of a hit ^[4]. The paper's account of why this went unmeasured is the more useful part. Retrieval-centric evaluation implicitly assumes stored information stays valid forever, while real interaction is non-stationary, with user facts updated, corrected or withdrawn over time ^[13].

The depth figures reward a little arithmetic. In LoCoMo, 94% of evaluation questions need grounding evidence from no more than two previous sessions ^[7]; the authors report the same pattern for 85% of LongMemEval questions ^[8]. That leaves at most 6% and 15% respectively reaching further back than two sessions ^[16][17]. Whatever "long-term" means in those suites, it is mostly one hop.

On mutation the ceiling is lower still. LongMemEval includes knowledge-update operations but caps them at two sessions before evaluation ^[10], and PersonaMem handles updates across no more than three ^[11], so the deepest revision chain any of these benchmarks asks a model to reconcile is three sessions long ^[18]. A production assistant that a user has corrected four times across five months is outside the tested envelope entirely.

The word carrying the weight in the results is "frequent". The abstract and introduction describe frequent reuse of invalid memories without putting a number on it ^[19], so nobody can yet rank the six systems by how often they act on a dead fact, which is the number a buyer would want. What makes the finding checkable rather than assertion is the build: the authors say data quality was controlled with automated memory-grounding checks plus human evaluation ^[3], and that code and data are published ^[15].

If it survives replication, the engineering consequence is not a larger index. It is a write path that can mark a fact dead, with supersession and a timestamp on the moment the user changed their mind. Statelessness was the problem the memory layer was sold against, since the cache holding short-term context is discarded when an interaction ends ^[14]. Retention without invalidation is a different fault with a similar symptom: an assistant answering confidently from information its user already retracted.