Build1 publisher3 min readPublished
An unquoted glob in a generated pack.sh makes LC_COLLATE part of the build
A dev.to walkthrough traces a checksum failure that only appears in CI to the shell expanding fragments/*.json in whatever collation order the job inherited, and proposes committing the file list so the order ships with the script.
The Engineer · Build desk
What happened
- A merge request passes on the author's laptop and fails in CI, where bundle.json has the wrong checksum and the job log still claims every fragment was included.
- The failing step is a generated pack.sh that concatenates fragments/*.json, and the shell expands that glob in collation order, so the job's locale sets the byte order of the bundle.
- Linux keeps Auth.json and auth.json as two files while default macOS APFS may not, so one tree can report a single path locally and two in CI.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint The file set never changes, so the ordered hash is the only assertion that fires, and it points the reviewer at the hasher; a pipeline step that verifies presence cannot see this defect at all.
- decision Committing the manifest moves fragment order into the diff, so every new fragment needs an ordering decision from a human reviewer instead of from whatever locale the runner booted with.
- exposure Any other generated step built on the same short glob is exposed, including concatenated OpenAPI documents and SQL migration chains, where two swapped entries change what the artifact means.
- cost The post ships templates it says have not been run, so the adopting team pays for the measurement: a host carrying both locales, and one run of the demo, before anyone trusts the diagnosis in a postmortem.
The first instruction in the post is to print the environment the job actually had: run `locale`, echo `LANG`, `LC_ALL` and `LC_COLLATE`, then run the same block on the machine that produced the patch, and do not argue from memory [17]. In the scenario the reviewer is on macOS, the agent wrote the script on Linux, and those are different inputs to the same script; the post observes that many hosted runners sit closer to `C.UTF-8` than to a desktop GUI locale [18].
The demonstration set is three files: `a.json`, `B.json`, `c.json` [6]. Under `LC_ALL=C` the expansion begins with `B.json`; under many UTF-8 locales it begins with `a.json` [5]. Two of the three paths change position and the third stays put [20]. The post leaves the resulting checksums unprinted. It tells you to watch them, and to keep the uppercase file if they match [23].
The divergence in that example rides on one capital letter [21]. The glob is unpinned either way, and the failing job arrives with the first uppercase fragment name. Take `B.json` out and both locales produce the same order [21].
The script is labelled an unexecuted template in its own comment, and the post says to treat it "as a demonstration template, not as a result you already measured" [8]. The checksums are the part you have to generate on your own hosts. The behaviour is documented: the shell expands the glob in collation order, and `en_US.UTF-8` and `C` do not sort the same byte strings [3][4].
The proposed `pack.sh` does two separate things. It exports `LC_ALL=C` and `LANG=C`, and it reads one relative path per line from a `fragments.manifest` kept in git, with comment lines allowed [11]. A missing manifest exits 2. A manifest entry that is not a file exits 3 [12]. A short bundle becomes a failed job with a path in the message.
Where a manifest is impractical, the post offers `find -name '*.json' -type f` piped through `LC_ALL=C sort`, and rates it above a raw glob and below a reviewed list [13]. It scopes that fallback to snapshot dumps and keeps it away from anything whose sequence matters [14]. One swapped pair is enough to poison a concatenated OpenAPI file or a SQL migration chain [19].
There is a second bug sitting next to the first. Linux keeps `Auth.json` and `auth.json` as two files, default macOS APFS may not, and an agent on a Linux box can create both, so the laptop reports one path and CI reports two [7].
On the runner, the post is blunt: if `locale -a` does not list `en_US.UTF-8`, do not install extra locales on a production host to imitate a laptop, and pin the script instead [16].
What to watch
- Whether agent-drafted fragments.manifest files get their order reviewed, since the post concedes a model may write the list.
- On a host where locale -a lacks en_US.UTF-8, the demo's first pack falls back to C.UTF-8, so matching checksums prove nothing about the bug.
- A runner base image that gains a desktop locale would change the bundle's byte order while pack.sh stays exactly as committed.