Build1 publisher3 min readPublished
Checking a revision register keeps superseded documents out of RAG answers
Three checks in a dev.to Python tutorial reject retrieved RAG documents that are superseded, not yet effective, or overdue for review. Authority sits in a register owned by the document process, and the clock is passed in, so every rejection is reproducible.
The Engineer · Build desk
What happened
- A dev.to tutorial describes an AI assistant that cites a real document and reproduces its instructions accurately, a week after that document was replaced.
- The fix it builds is a gate of three independent checks that compares each retrieved document with an approved-revision register and explicit time rules before generation.
- The example is a single Python 3.12 file, check_freshness.py, that needs only the standard library.
- The tutorial states that the gate cannot prove an accepted document is factually correct.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure Whoever maintains the revision register now decides what the assistant may cite. If the register lags a replacement, the old revision passes the gate with a clean result.
- decision A team that also answers questions about past versions of a policy needs a second selection policy, because this gate rejects every superseded revision on purpose.
- constraint Revisions are compared as exact strings, so the register and the document metadata need one canonical label format, or current documents get rejected as superseded.
A retriever ranks a passage by how well it matches the question. It can rank a passage highly even when that passage's lifecycle metadata makes it unsuitable for a current answer [3]. The tutorial's author wrote: "The citation proves that a source exists; it does not establish that the source is still approved for the current question." [2]
A call to `check_document(doc, current_revisions, as_of)` runs in four steps:
1. It validates the metadata. The ID and revision must be non-empty strings, both timestamps must parse with a timezone, and `review_by` must fall after `effective_at`. Any failure returns `invalid_metadata` with `allowed` false [7]. 2. It looks the ID up in the register. A missing ID adds `unknown_document`. A revision that differs from the registered one adds `superseded_revision` [8]. 3. It compares `as_of` with the document's dates. Before `effective_at` adds `not_effective_yet`. At or past `review_by` adds `review_overdue` [9]. 4. It returns every reason it collected, so one document can be superseded and overdue at once [10].
Step 1 fails closed. Bad metadata stops at the gate instead of falling through to approval [7]. Step 4 returns named reasons in place of one unexplained Boolean [10]. An operator reading a rejection can see which rule fired.
The demo fixes `as_of` at 08:00 UTC on 25 September 2026 and registers revisions 3, 2, 5 and 1 for four notes [13]. Traced through the code, the fixture resolves like this. `note-a` holds its registered revision and sits inside its window, so it passes. The second note carries revision 1 against a registered 2 and fails as superseded. The third passed its review date on 20 September. The fourth does not take effect until 1 October [14][15]. One of the four documents reaches the generator [15].
The time handling is the best engineering in the tutorial. The window is half-open: a document becomes eligible at its effective timestamp and overdue at the exact review deadline [9]. Evaluation time is passed in. Reading the wall clock inside the function would make a saved fixture behave differently as time passes, while a fixed `as_of` keeps the test reproducible [11]. Timestamps are normalized to UTC, and one without timezone information is rejected instead of guessed at [12].
Authority stays outside the retrieved text. The model may not infer the current revision from a filename or from a passage saying "this is the latest version" [16]. A passage that announces it is current is just more retrieved text. The newest timestamp does not settle the question either. The tutorial leaves it to the source owner to decide which revision is approved [17].
The review date gets the same restraint. In the tutorial, "review overdue" means the chosen process requires attention, and a fact does not become false at midnight [18].
What to watch
- A rejection rate measured on a production corpus, showing how often top-ranked passages fail the gate; the four-note demo only shows that each rule fires.
- Retrievers that filter on revision and effective dates at query time, so stale documents never enter the ranked set at all.
- Document-management systems publishing an approved-revision feed that a gate like this can read directly.