If two people cannot reproduce the number, the dashboard is decoration

An operational design guide published on dev.to argues that the recurring failure in security reporting is not the charting tool: a dashboard can look precise while hiding the fact that two people cannot reproduce the same number from the same evidence ^[1]. Its diagnosis is an undefined metric contract, where scope, formula, denominator, source, timestamp, owner, validation and exception handling sit in different notes or in nobody's notes ^[2]. The worked failure is a tile named "Critical vulnerabilities remediated on time" ^[3]. The guide lists five things it could mean: closed findings over all findings created this month; findings closed within SLA over findings due this month; affected assets fixed over affected assets discovered; scanner instances marked resolved even when exceptions are open; and production assets only versus every inventoried asset ^[3]. All five return a percentage and none answer the same question ^[4]. They also diverge on at least four axes at once, namely the denominator window, the counting unit, the treatment of open exceptions, and asset scope ^[5]. Attaching a target does not resolve that; the guide's position is that the denominator and inclusion rules must be explicit first ^[6]. The replacement artifact is one reviewable record per metric, with a stable ID, captured before the metric enters a recurring report ^[7]. The example record is dense and boring in the right way: ID VM-SLA-CRIT-001, display name "Percentage of due critical findings remediated within approved SLA", a decision question ("where should remediation escalation focus next week?"), a population of critical findings whose SLA due date falls in the period on in-scope production assets, a numerator of those with a validated remediation timestamp on or before the due date, and a denominator of the population with approved risk acceptances broken out as a separately labeled exception count ^[8]. The formula is numerator divided by denominator times 100, to one decimal place, with a zero denominator returning N/A rather than 100 percent ^[9]. That last rule carries more weight than it looks: with an empty population the ratio has no value, and defaulting to 100 would report perfect performance in a period where nothing was due ^[10]. The rest is timing and proof. The as-of is 23:59 Asia/Seoul on the final day of the period, with scanner export and ticket snapshot no older than the declared cutoff ^[11]. Validation runs before calculation and covers schema, required fields, allowed values, timestamps, uniqueness, joins and reconciliation of aggregates to source totals ^[12]; the guide cites NIST for the point that checking data integrity, accuracy and structure before analysis addresses potential errors ^[13]. NIST, as the guide reads it, also defines data sources broadly enough that "from the dashboard" is not a sufficient source description ^[14], and treats scope as what distinguishes total risk from the risks currently being measured ^[15]. The acceptance test is the part most programs skip: hand the definition and a frozen input extract to a second reviewer, and the metric passes only if that reviewer reproduces the published value within the stated rounding rule and records the same exceptions ^[16]. Published values then carry period, as-of timestamp, definition version, scope note, source freshness and quality status ^[17]. When population, formula, severity logic, source or exception policy changes, the guide says do not overwrite: close the old version with an end date, open a new one with an effective date and change reason, label any backfill explicitly, keep old dashboard values linked to the definition in force at the time, and run both versions in parallel for one review cycle ^[18]. Two things to watch in your own pack. Whether a quiet month renders as N/A or as a suspiciously clean 100 percent, and whether the next definition change produces a version with an end date or a silently edited tile.