Loading today’s stories
Build1 distinct publisher3 min readUpdated
An engineer deploying SigNoz into an isolated Docker network reports that ClickHouse 25.5.6 rejects the old config mount, and the OTel collector will not boot before a schema migrator has finished.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
SigNoz's official repository still presents self-hosting as a single `docker-compose up` away, a ten-minute job before you get back to business logic [1]. According to a dev.to write-up by an engineer who built the observability stack for Verne Software, that promise now breaks on contact with the current images, because the documentation lags behind architectural shifts in the underlying Docker images [2][3].
The first trap is ClickHouse. On 25.5.6, mounting your own `clickhouse-config.xml` over the default, exactly as older tutorials instruct, leaves the server refusing to start, with errors about missing paths or invalid top-level settings [4][5]. The author reports a specific crash loop: `Code: 137. A setting 'log_queries' appeared at top level in config. But it is user-level setting that should be located in users.xml inside <profiles> section.` [6]. The fix is to stop replacing the main config and instead drop overrides into `config.d/` and `users.d/`, with query logging declared inside a user profile in a `users.xml` mounted at `/etc/clickhouse-server/users.d/users.xml` [7][8]. The author's dividing line: cluster topologies and UDF paths belong in `config.d/`, profiles and passwords in `users.d/` [9].
The second trap is that schema creation is now strictly decoupled from application logic [11]. Start ClickHouse and the OpenTelemetry collector together, as you could in older iterations, and the collector crash-loops with `Database signoz_traces does not exist` [10][12]. Three databases are involved, signoz_traces, signoz_metrics and signoz_logs, and none of them will exist unless something creates them first [12][13].
That something is a pair of ephemeral containers, which is the real change in shape here. The author's working boot sequence runs `signoz_init_clickhouse`, a one-shot container that downloads UDF binaries such as `histogramQuantile` [14], then ZooKeeper as ClickHouse's coordinator [15], then ClickHouse itself [16], then `signoz_telemetrystore_migrator` to handle schemas [17], and only then the collector and the SigNoz API and UI [18]. Two gates enforce it: the migrator waits on ClickHouse with `condition: service_healthy`, and the collector waits on the migrator with `condition: service_completed_successfully` [19][20]. Two of those six components exist only to run to completion and exit before any long-running consumer is allowed to start [21].
Networking is the third element, and the published excerpt is thinner here. The stack sits in an isolated `verne_observability` network, with the SigNoz API and UI container additionally attached to an internal `verne_internal` network so the core API can proxy dashboard requests without exposing the database or internal collector ports outward [22][23]. The author lists networking quirks among the traps encountered, alongside ClickHouse configuration and OpenTelemetry bugs, but the material available stops before that section is worked through [24].
What to watch: whether the upstream compose file grows these `depends_on` conditions, and whether the ClickHouse image tag in your own deployment floats. A stack whose correctness depends on ordered one-shot containers and on which directory a setting lives in is a stack where an unpinned minor version upgrade is a production incident. Treat the quickstart as a demo and the compose file as something you own.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
signoz_init_clickhouse is a one-shot container that downloads essential UDF binaries such as histogramQuantile.
signoz_zookeeper is the coordinator for ClickHouse in the stack.
signoz_clickhouse is the main datastore.
signoz_telemetrystore_migrator is described as a crucial bootstrap step that handles schemas.
signoz_otel_collector and signoz (the API and UI container) are the final data consumers in the boot order; in the modern stack you can no longer start everything at once and order matters immensely.
In the author's docker-compose.yml, signoz_telemetrystore_migrator declares depends_on signoz_clickhouse with condition: service_healthy, to wait until the database can accept migrations.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed but single-source and self-reported
The technical detail is unusually specific for a single post — an exact ClickHouse crash string (Code: 137, log_queries), named databases, a six-service boot order and verbatim compose snippets — which makes the failure modes checkable by anyone reproducing them. But the entire cluster is one dev.to article by the engineer involved, with no maintainer confirmation, upstream changelog, issue link or second deployment, and the text truncates mid-section. That caps evidence below the midpoint.
One disclosed self-host deployment
Adoption evidence amounts to a single first-party deployment: the author's Verne Software stack, with a pinned collector image tag. There are no user counts, download figures, other organizations, or vendor usage disclosures in the supplied material, so the measured signal is real but minimal.
Concrete fixes, over-general headline
The individual technical claims are modest and well specified, so there is little inflation at the detail level. The generalization that self-hosting is 'no longer a compose file' does run ahead of the evidence: it extrapolates from one uncorroborated deployment on specific image versions, with no upstream confirmation that the quickstart is broken for others. That yields a small positive gap rather than a large one.
Practitioner promotion, no vendor tie disclosed
The author writes as the architect of Verne Software's infrastructure and names that company in the post and network naming, giving a mild personal- and company-visibility incentive typical of developer-platform publishing. Nothing in the material indicates a paid relationship with, or opposition to, SigNoz, ClickHouse or a competing vendor, and the content is critical of a tool he continues to run, which limits distortion.
Credible first-hand account, unverified breadth
Confidence is moderate: the operational specifics hang together and are internally consistent, and the fixes follow from the errors described. What is unverified is generality — whether these breakages affect the current upstream quickstart broadly, on which versions, and for how long — because only one publisher and one deployment are in evidence and the article text is truncated.
leadership
ClickHouse buys Langfuse, turning a neutral tracing layer into someone's roadmap1 distinct publisher
product
OpenTelemetry is free; the collector fleet, the retention policy and the on-call rota are not1 distinct publisher
build
Zero-Instrumentation Go Tracing Works, But You Are Now Maintaining ABI Offsets1 distinct publisher
build
Three boxes, seven hops: a container diagram is not an incident map1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 19, 2026