Product1 distinct publisher3 min readUpdated
A CNCF guide argues the answer to slow-query incidents is not more collection but traffic-weighted metrics derived from database spans. The implication for observability budgets is the interesting part.
The Product Desk · Product desk
Compiled by The Product DeskSomething wrong?How this is made
A guide published on the CNCF blog opens by rejecting the reflex response to slow SQL: the traditional fix is to collect more telemetry, and more telemetry means more things to look at, not necessarily more understanding [1]. Its alternative is narrow enough to be testable, which is why it matters: take OpenTelemetry database spans and distil them into span-derived metrics you can put on a dashboard and alert on [2].
The framing worth stealing is the phrase "the moment of decision." The authors argue teams should be opinionated about what matters then, rather than treating traces as a stream someone might analyse someday, and that raw telemetry only becomes useful once meaningful patterns are extracted from it [3]. That argument is developed in a companion talk by Endre Sara on managing telemetry overload [4]. Collection is the cheap part of the pipeline to buy and the expensive part to keep. Decision is the part nobody budgets for.
The guide reduces the problem to two questions: which queries yield the most value if made faster, weighted by traffic, and which queries are behaving abnormally right now [5]. The lab it describes is staged in that order, from simple slow query detection, to traffic-weighted impact, to anomaly detection [6]. Note what is absent: a general-purpose query explorer.
Traffic weighting earns its place because "slow" is not one number. A 50ms query can be fine for a reporting dashboard and catastrophic for checkout [7], and the guide catalogues causes that need different fixes: excessive work from missing or unusable indexes, resource contention from locks and connection pool exhaustion, environmental pressure such as CPU saturation and I/O bottlenecks, and plan regressions from parameter-sensitive plans or stale statistics after bulk loads [8][9][10][11]. Its scaling example is the one to keep: without an index on customer_id, a single-predicate SELECT on orders goes from 20ms at 10K rows to minutes at 10M rows, with the query unchanged and only the data volume moving [12], a thousandfold increase in rows [13]. A query spending 95% of its time waiting on locks is not fixed by query optimisation, it needs transaction redesign [9].
Then there is the case that breaks threshold alerting outright. The N+1 pattern runs 100 queries at 2ms each in sequence, adding 200ms plus network overhead, and no individual query is slow [14]. Any slow-query threshold above 2ms logs none of those 100 queries while the user waits 200ms [15]. Databases ship good tools here, including slow query logs, pg_stat_statements and EXPLAIN, but they report what is expensive inside the database, not which service triggered it, whether it is user-facing or background, or whether it correlates with the latency spike under investigation [16][17]. The usual bridge is a human one: a developer notices a slow endpoint and takes the query to a DBA [18]. The guide's claim is that this manual linking is the automatable part [18].
What to watch: whether the traffic-weighted metric actually displaces that developer-to-DBA handoff, and whether the anomaly detection stage, which the guide sequences last, survives contact with real traffic patterns [6][18].
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
The traditional fix for slow SQL queries is to collect more telemetry, but more telemetry means more things to look at, not necessarily more understanding.
The guide describes a repeatable workflow that turns OpenTelemetry database spans into span-derived metrics you can dashboard and alert on, to identify what is slow, what matters most, and what just regressed.
Rather than treating traces as a data stream that might be analysed someday, teams should be opinionated about what matters at the moment of decision; raw telemetry only becomes useful when meaningful patterns are extracted.
The post embeds a video titled "The Signal in the Storm: Practical Strategies for Managing Telemetry Overload" by Endre Sara.
The workflow serves two use cases: optimization (which queries yield the most value if made faster, weighted by traffic) and incident response (which queries are behaving abnormally right now).
The lab has the app emit OpenTelemetry traces which are distilled into actionable metrics, starting with simple slow query detection, then adding traffic-weighted impact, and finally anomaly detection.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Mechanism well described, effectiveness unmeasured
The causal taxonomy of slow queries and the gap in database-native tooling are clearly argued and internally consistent, and the pipeline is concrete enough to reproduce. But every quantity in the piece is illustrative (20ms at 10K rows to minutes at 10M rows; 100 queries at 2ms adding 200ms), there is no measurement that the three span-derived metrics detect regressions or reduce triage time, and only one publisher is in the cluster.
No adoption disclosed beyond the tutorial's own lab
The only implementation described is the guide's self-authored demonstration lab. The cluster discloses no production deployments, user counts, downloads, benchmark results or organisational usage of this span-derived-metrics workflow, so adoption cannot be scored without inventing facts.
Mildly overstated relative to demonstrated results
The prescriptive framing — that three derived metrics are what slow queries actually need, and that manual developer-to-DBA linking 'is exactly what we can automate' — runs ahead of what a single lab walkthrough demonstrates, and the cluster dek pushes an observability-budget implication the article never quantifies. The gap is modest rather than large because the underlying claims about query slowness and missing request context are uncontroversial and the guide is explicit that it is building a lab.
Foundation promoting its own project and a bundled vendor stack
CNCF publishes the guide and hosts OpenTelemetry, the technology the guide recommends; the lab is built on Grafana's docker-otel-lgtm bundle, and the post embeds and back-references a talk by a named speaker. That is a clear ecosystem-promotion incentive, though it is disclosed in plain sight and the content is a technically substantive how-to rather than a product pitch.
Low-to-moderate: one interested publisher, no corroboration
Claims about the article's contents are fully verifiable from the supplied text, so descriptive confidence is high, but the substantive assertion — that span-derived, traffic-weighted metrics beat collecting more telemetry — rests on one source published by an interested foundation with no adoption evidence and no measured outcomes.
product
Sovereignty audits are moving from the region picker to the plane topology1 distinct publisher
product
A hyperscaler reportedly handed Dutch regulators' emails to Congress. Residency was never sovereignty.1 distinct publisher
product
OpenTelemetry is free; the collector fleet, the retention policy and the on-call rota are not1 distinct publisher
build
Notion's agent stack is live, not slideware, and it only changes one of your decisions1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 21, 2026