Build1 distinct publisher3 min readPublished
Nobody had read the scanner's scope since install, so it was billing against every log line plus APM spans and RUM sessions. Scoping it took an afternoon, and the same review then found the same kind of drift in two other places.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Start with the meter. A scanning group's cost is a function of how much data enters its scope, and scope is one field in the config. Before the fix that field held no filter, so the group covered every log in the org plus APM spans and RUM sessions from sources where, by the author's account, PII could not appear [4]. The feature was doing exactly what it had been configured to do on the day someone switched it on, and that configuration is what generated the bill.
Check the arithmetic before you borrow the number. $390 a day down to $56 a day is $334 saved, and $334 is 85.6% of $390, not the 87% in the writeup [6][1]. Annualise the daily delta and you get $121,910, which is where the reported $120,000 figure comes from [2]. Neither rounding changes the decision, but if you are carrying this to a finance partner, carry the $334 a day.
The saving transfers only if your scanner bills on volume scanned rather than per source, and only if most of your log volume comes from services where regulated fields genuinely cannot land. Three services made the cut here: checkout, onboarding, and the user API, in prod [5]. A payroll processor plausibly does concentrate its PII behind a handful of customer-facing entry points [2]. If yours is spread across every service, that filter buys you nothing and the money is somewhere else.
The ghost services are the sharper failure. A shared logging wrapper called tracer.set_tags with a literal service name [7], and every application that imported it published that name alongside its own [8]. Datadog then treated each phantom as real, so monitors and coverage-audit entries multiplied and an inflated service count fed incident routing [9]. The correction reads identity from DD_SERVICE at the edge instead [10]. It had to ship into more than 30 production repositories, because a hard-coded tag inside a dependency reaches every importer [11]. That is the pattern in this bug class. The blast radius equals the dependency graph, not the size of the diff.
The index leak is the one I would put on a recurring schedule. A group-by on @env against index:main-prod returned 66.1M production events and 24.5M staging [12], and 24.5 divided by 90.6 is 27% [3]. The first read of the quota breach was organic growth; it was the author's manager who looked again and found a 13M day-over-day jump on two services [13][14]. Log indexes are routing rules. Routing rules drift when nobody diffs them [18].
The scanner accounts for $120,000 of the reported total, which leaves at least $30,000 a year across the other two findings, unpriced individually [4]. All three are config files with no reviewer [17]. A codebase usually gets a linter and a second pair of eyes checking changes before they ship. A scanning group typically just gets an install and then a monthly invoice, with nobody assigned to look at it again.
Ranked by verification strength, evidence, and original report placement.
Daily cost dropped from $390 to $56, which the author describes as an 87% reduction and $120,000/year in savings from a config change that took an afternoon.
A Datadog log analytics query grouping index:main-prod by @env returned env:production 66.1M and env:staging 24.5M.
A staging application load balancer was routing 24.5 million access logs a day into the production log index, which the author calls 27% of that index's volume and pure waste against quota that existed to protect production observability.
In one month the Sensitive Data Scanner cost reached $390 per day, which the author states as $11,700 per month.
The scanning group had no filter, so scope included every log plus APM spans and RUM sessions, including data sources where PII could not possibly exist; the default config scanned it all and nobody had revisited the scope since initial setup.
The fix was a scoping change on scanning group pii-scan-prod adding filter query "service:(checkout-api OR onboarding OR user-api) env:prod" with is_enabled: true, so only logs matching the query are scanned.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · September 3, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
The three AWS defaults on your bill that nobody actually chose1 distinct publisher
build
Under 30% citation overlap between engines makes pooled AI visibility scores unbuyable1 distinct publisher
build
CAPTCHA walls in Playwright agent pipelines are usually self-inflicted retry bursts1 distinct publisher
build
Eight containers against a variable APM bill: what the self-hosted stack actually costs you1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Receipts in config, not in billing
Everything here rests on one engineer's account: no invoice, no employer, no colleague who saw the before-and-after. What lifts it above anecdote is that the post shows its mechanics — the scanning group filter it added, the tracer call it replaced, the query that returned 66.1M production events against 24.5M staging ones. Those are reproducible in any Datadog account, which is not the same as having been reproduced here. And the one number a reader can check independently, the 87% cut, is wrong by a point and a half.
One unnamed shop, three fixes shipped
Real changes appear to have landed — a re-scoped scanning group, a tagging fix across 30-plus repositories, an exclusion filter on the production index — but all inside one organisation that is never named, on the word of the person who made them. Nothing indicates anyone else has run the same audit or found the same drift, and the four-layer framework at the end has exactly zero users outside the author.
Round numbers doing unearned work
The mechanism is entirely credible — volume-priced features default to capturing everything, and nobody re-reads scope after install. The packaging is where it stretches. A $150K+ total with $120K accounted for, an 87% cut that is 85.6%, and a headline saving that survives only if $56 a day holds for a full year, which no follow-up month confirms. Overstated at the edges, not at the core.
A war story that ends in a framework
The shape is familiar and mostly benign: establish scale (payroll for hundreds of thousands), produce a six-figure number, then arrive at "a FinOps framework I wish I'd had from day one." No vendor appears to be involved and the story is unflattering to the author's own past configuration, including a misdiagnosis their manager had to correct. The pressure worth naming is reputational rather than commercial — bigger savings and rounder percentages make the better post, and that is exactly where the numbers drift.
Trust the pattern, verify the total
The failure modes described are the kind experienced operators recognise on sight, and the configuration artifacts make them concrete enough to act on. What deserves no confidence is the accounting: a single unverified voice, one arithmetic slip already visible, and a year-long total whose remainder is unexplained. Read it as a checklist worth running, not as a benchmark worth citing.