Build1 publisher3 min readPublished
logq's -j flag shards aggregation by hash of the group key and was proved byte-identical at sixteen workers, but nobody proved it faster, and the team published the losing number instead of holding the flag back.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Hash routing removes the usual reason a parallel aggregation disappoints, which is lock contention on one shared map [4]. It adds two costs it cannot remove: a hash computed per record on the routing path, and a handoff per record across a goroutine boundary. On a counting workload, those two costs are most of the per-record work.
The published run is where that shows. One worker moved 11.7MB per second and eight moved 10.1MB/s [15], a wall-clock gap of 1.02 seconds [13] and a speedup of 0.86x, roughly 11% of what eight workers would give if the stage scaled linearly [18]. Neither throughput figure is fast for grouped counting, which points at the stage in front of the shards: logq's decoder builds the full ordered record for every line, whatever fields the query names [9]. Sharding behind a serial bottleneck buys coordination and no throughput. That is an inference from the decoder's contract rather than a profile, because the post reports elapsed times without a core count, a query, or a run count [16].
For -j 8 to win on someone else's logs, the per-record work inside a shard has to be big enough to pay for the hash and the handoff. logq computes percentiles as well as counts [3], and a percentile over many groups does more per record than a counter increment does. Cardinality has to cooperate too, since a group is pinned to one shard by its hash [4]: a single dominant group means one busy goroutine and seven idle ones. The equality check also ran at 16 workers over 2,000 records while the timing ran at 8 workers over 76.3MB [5][1], so neither measurement covers the other's conditions.
The commit order tells you what this team tested first. The three-valued MISSING/null/false evaluator and its generated truth-table tests land 28 minutes into the build, before the JSON decoder and before the CLI existed [10]; the decoder that replaces gjson lands four minutes later [7]; the parallel flag arrives about 23 hours after that [17]. The replacement targets two stdlib defaults precisely: every JSON number decoded to float64, which loses precision past 2^53 and hands back a confidently printed wrong trace ID with no error, and Go's undefined map iteration order [8]. BENCHMARKS.md records the case where the swap loses, a query reading one field out of thirty [9], and the hashes are in the post because the project's own dependency ledger treats the git log as the evidence trail [12].
A commit hash lets a reader re-run the correctness claim. The speed claim is the one thing here that has to be re-measured on the reader's own file and cores, and until someone does that, 16% describes a 76.3MB file and the machine it ran on [1][2][14].
Ranked by verification strength, evidence, and original report placement.
logq's -j 8 flag (8 parallel workers) finished a query against a real 76.3MB file in 7.558 seconds; the same query on the same file with one worker finished in 6.535 seconds.
Commit a29a587, twenty-three and a half hours into the project, shards the stats command's per-group aggregation across N goroutines, routing by a hash of the group key so a group's records always land on the same shard, with no round-robin and no cross-shard merge beyond a trivial flush at the end.
The post states that eight workers lost to one by 16%, and that the flag shipped anyway, with the number disclosed the moment it was found.
logq is a command-line log query tool that can filter, group, count, percentile and time-window a JSONL/logfmt/plain-text file, shipped as one static binary with zero third-party dependencies, built for Track B (Parsers & Data Formats) of the Zero Dependency Hackathon.
The same commit verified correctness by showing byte-identical output between -j 1 and -j 16 over a 2,000-record fixture; speed was not verified.
tidwall/gjson has 10,420 known importers on pkg.go.dev, a figure the authors checked live twice, a day apart.
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Checkable in the repo, thin as a measurement
This write-up does something project posts rarely do: it names the commit that caught the regression (58da5cf) and prints both timings to the millisecond, so anyone with the public repo can trace the history behind each assertion. What it never prints is the machine's core count, the query text, or whether either run happened more than once, and 7.558 against 6.535 seconds is a narrow enough margin that one unrepeated pair cannot carry much on its own. The 80/20 split between decode and aggregation, which is the whole causal explanation, has no profile attached.
One hackathon entry, no users named
The measured deployment surface is the authors' own machine. logq is a Track B submission to the Zero Dependency Hackathon, and the post names no downloads, installs, contributors or production use. The one real usage figure in the story belongs to the library that was removed: gjson's 10,420 known importers, which is a count of other people's adoption, not logq's.
Leads with its own worst number
The headline figure is the one that makes the work look bad, disclosed in the same commit that produced it, and the concession that gjson still wins on single-field extraction sits in the project's own benchmarks file. That is the reverse of the usual optimisation post, which is why the gap runs negative. It is not fully negative: the claim that decoding dominates per-line cost and the promise that a future version sharding decode will make -j worth having both go further than anything actually measured here.
Contest entrant writing up their own entry
Dev.to's post is self-published by the people whose hackathon submission it describes, and visibility for that submission is part of what the post buys. The disclosed regression works against a purely promotional reading, though a candid loss in a portfolio piece earns credibility of its own, so honest disclosure and self-interest point the same way here. No sponsor, employer or funding relationship is named anywhere.
Plausible and checkable, not verified
Every figure has a route to confirmation: hashes, file size, line count and timings all point into a public repo, and the authors say they re-read the pkg.go.dev count a day later so it would not be stale. It remains one team's self-report on a project built over a few days, with the CI that might have backed it deleted before submission, so the arithmetic on the two timings holds firmly while the causal story behind them rests on the authors' word.
build
Rust's from_str waves through a tenth of the JSON that RFC 8259 says to reject1 publisher
build
A pure-Go DICOM stack takes CGO out of the hospital build pipeline1 publisher
build
Kimi Desktop's second binary: mutable CDN path, no Windows checksum, no signer check1 publisher
security
Cambodia-focused Spark RAT campaign turns OPSWAT's AppRemover driver into an EDR killer2 publishers
Publishers with included, body-backed reporting in this cluster.
1 article · September 6, 2026