Build1 distinct publisher3 min readUpdated
Lakshman Pandey's write-up maps a production retrieval system to all four NIST functions. The controls that exist are documents and evals; the ones that would gate a deploy are still marked future.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
A developer has published a control-by-control mapping of a live retrieval-augmented generation system to the NIST AI Risk Management Framework, for a system serving UK arts and culture clients [1]. The line that matters to anyone being quoted an enterprise governance platform: the whole thing runs on $25 a month of infrastructure plus $0.003 to $0.005 per query [2].
The framework has four functions, GOVERN, MAP, MEASURE and MANAGE [3]. What fills them in this build is paperwork and instrumentation, not products. GOVERN is an architecture decision record, ADR-001, which records the choice of Supabase pgvector in EU-West-2 (London) because UK public-sector cultural clients require UK/EU data residency for GDPR, names the resulting vendor dependency as the risk, and cites the eval framework plus the ADR itself as the reversibility mitigation [5]. The stated policy is that user data stays in the EU, with calls to Claude and Voyage treated as transient, and Langfuse holding an audit trail of each query's origin and destination [6]. MAP is an inventory over a corpus of 108 documents, rated LOW-RISK on the grounds that the system is retrieval-grounded, the corpus is small and controlled, stakeholders are few, and no real-time safety-critical decisions are involved [7].
Note who signs that rating. Decision authority is described as a solo architect with client stakeholder approval loops, the client approves governance policies and validates output quality, and the operations role is listed as TBD [8]. This is self-assessment with a customer in the room, which is what most small vendors actually have.
MEASURE is the part worth copying. There is a Ragas-based suite of 18 golden questions scoring faithfulness, context precision, context recall and answer relevancy, chosen respectively to catch hallucination, irrelevant retrieved chunks, missed chunks, and off-question answers [9]. Langfuse traces every production query; the sample trace shows 450 input tokens, 85 output tokens, $0.0031 and 1250 ms [10]. Cost per query is broken out as $0.0005 for embedding and $0.003 for generation [11], so generation is roughly 97 percent of marginal cost [4]. The latency target is under two seconds against about 1.2 seconds observed, about 40 percent inside the target [12][5].
MANAGE is thinner, and the author does not pretend otherwise. Today it is a system prompt constraining answers to the supplied sources and try-catch handling that logs errors to Langfuse [13]. Rate limiting and a monthly cost ceiling are marked future [14], as are a human-in-the-loop gate for sensitive queries, prompt caching claimed to cut costs 25 to 50 percent, Presidio PII redaction, and a CI/CD regression gate requiring the eval suite to pass before deploy [15]. Until that last item lands, the eval suite is a report, not a gate.
The economics explain the architecture. At $0.0031 a query, the $25 fixed monthly cost dominates until roughly 8,000 queries a month [2]; 10,000 queries costs about $56 all in [3], and the floor is $300 a year [1]. Managed Supabase was chosen for backups, EU residency and zero DevOps, at the price of lock-in and a moderate migration cost, on the reasoning that a team of one has no DevOps capacity and clients demand EU residency [16].
Watch whether the regression gate and the cost ceiling ship, and whether an operations owner is named. Those three are the difference between a documented posture and an enforced one.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
ADR-001 records the decision to use Supabase pgvector in EU-West-2 (London) because UK public-sector cultural clients require UK/EU data residency for GDPR compliance; the logged risk is vendor dependency on Supabase, and the mitigation is that the eval framework plus the ADR ensure reversibility.
Policy states all user data stays in the EU and API calls to Claude and Voyage are transient with no data stored in the US; measurement is a Langfuse audit trail logging every query's origin and destination.
The risk inventory gives an overall rating of LOW-RISK, justified by the system being retrieval-grounded rather than generative-primary, a small controlled corpus of 108 documents, limited stakeholders (internal plus client team), and no real-time safety-critical decisions.
Decision authority is a solo architect with client stakeholder approval loops; the developer handles system architecture, data pipeline and deployment, the UK public-sector client approves governance policies and validates output quality, and the operations role for monitoring and alert response is listed as future (TBD).
A Ragas-based evaluation suite with 18 golden questions measures faithfulness (detects hallucination), context precision (retrieved chunks are relevant), context recall (good chunks are not missed) and answer relevancy (answer matches the question).
Langfuse traces every production query; the sample trace records 450 input tokens, 85 output tokens, cost_usd 0.0031 and latency_ms 1250.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed but wholly self-reported
The disclosure is unusually concrete for a single blog post — an ADR with risk and mitigation, a JSON Langfuse trace with tokens, cost and latency, a quoted app.py snippet, a named model version, and an itemised per-query cost breakdown. But all of it comes from one self-published source authored by the system's own builder, with no independent audit, client attestation or provider confirmation, and the central MEASURE control (the Ragas suite) is described without any published scores, thresholds or run cadence. That supports architecture and unit-cost claims while leaving quality and compliance claims unverified.
One solo-built deployment, one client scope
Adoption evidence is real but minimal: one production system by a team of one, a 108-document corpus, stakeholders limited to internal plus a single client team, no operations owner, and no disclosed query volume, user count or contract value. Scaling to 10 clients is stated as a design property, not an observed fact. This is a working deployment rather than any indication of uptake of the pattern by others.
Compliance framing outruns enforced controls
The framing — a title claiming a NIST AI RMF-compliant system and a closing 'COMPLIANT with NIST' status with three of four functions ticked — sits ahead of what is actually enforced. Every control that would bind behaviour or gate a change is future: rate limiting, a per-client spend ceiling, human-in-the-loop review, PII redaction, and the CI/CD gate that would require the eval suite to pass before deploy. Reversibility from the vendor lock-in risk is asserted via an ADR plus evals rather than demonstrated. The architecture and cost claims, by contrast, are specific and internally consistent, which keeps the gap moderate rather than severe; the author also visibly self-flags the MANAGE shortfall.
Practitioner marketing regulated-domain credibility
The author is the builder, the assessor and the beneficiary of the assessment. The piece self-certifies compliance for a service sold into UK public-sector and cultural clients, where documented governance and EU residency are procurement gates, so a favourable NIST mapping is directly commercially useful to the writer. It is also a personal-brand artefact on a developer platform. Mitigating factors: the article volunteers its own MANAGE weaknesses, lists unbuilt controls, and discloses vendor lock-in — disclosure patterns inconsistent with pure promotion. No sponsorship, vendor payment or affiliate relationship is disclosed in the material.
Moderate on build facts, low on outcomes
Confidence is reasonable for what the system is made of and roughly what a query costs — those figures are itemised, internally consistent between the cost breakdown and the sample trace, and cheap to state falsely at little gain. Confidence is low on quality, compliance and durability: no eval scores, no aggregate telemetry, no third-party verification, one publisher, and a bylined date (August 2026) matching the publication date with no follow-up reporting. Derived economics inherit the source's own inputs and assume the disclosed prices hold.
build
Microsoft ships an MIT-licensed agent kernel: policy rings, Ed25519 identity, kill switch1 distinct publisher
build
Two Next.js apps instead of one, because isAdmin is a privilege escalation waiting to happen1 distinct publisher
build
Claude's system prompt grew ninefold in two years. Version yours like code.1 distinct publisher
build
A RAG Pipeline in 200 Lines of TypeScript, and the Parts the Frameworks Hide1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 18, 2026