Skip to content

Build1 publisher3 min readPublished

A decorative checkmark crashes pytest on a runner that boots with LANG=C

The crash happens inside the reporter while it renders the failure message, so the message you need is the one that fails to print. Four conditions have to hold for it to reach your CI. A UTF-8 laptop hides all of them.

The Engineer · Build desk

Illustration accompanying A decorative checkmark crashes pytest on a runner that boots with LANG=C

What happened

  • A dev.to lab notebook records two days spent on a pytest collector crash that never reproduced on the author's Mac, despite the same pinned CPython minor version and the same requirements lock.
  • On images that boot with LANG=C or LANG=POSIX, Python reports sys.stdout.encoding as ANSI_X3.4-1968, and pytest then fails while rendering a checkmark rather than while evaluating the assertion.
  • The post's diagnostic runs one script three ways on the same interpreter: unmodified, then under LANG=C LC_ALL=C, then with PYTHONUTF8=1 added to the C locale.
  • The article discloses that it was prepared as part of MonkeyCode's product outreach, using the vendor's free model access for patch ideas and its free server as a locale-clean Linux shell.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint When the abort happens inside the writer, the failing test's message is the thing you lose, so the cheapest diagnostic signal is destroyed by the defect it would have described.
  • decision Teams now choose between patching the runner environment and editing what test messages are allowed to contain, and only one of those choices leaves the reporter able to print the next contributor's characters.
  • exposure A formatting choice in an assertion string becomes a build-breaking input on any non-TTY runner, and a reviewer reading the diff on a UTF-8 laptop has no way to see it.
  • contradiction The mechanism is reproducible in four lines, but the claim that many cloud images boot at the C locale rests on one person's 48 hours with no image named, so prevalence is yours to measure.

The writer is what breaks here. In the notebook the interpreter raises UnicodeEncodeError while it is printing the failure message, before it ever decides whether the assertion passed [6], and the traceback points at a strict ASCII codec inside `safe_str` [7]. That is why the early guesses missed: pytest-xdist, color output, and a plugin suspected of caching a codec on stdout all sit above the layer that broke [8].

Reaching your pipeline requires a specific combination. The image has to leave the locale at C or POSIX, which resolves `sys.stdout.encoding` to ANSI_X3.4-1968, a long spelling of ASCII [4][5]. The process must not be a TTY, because no terminal emulator negotiates UTF-8 on behalf of a process that is not one [11]. Something non-ASCII has to reach the writer, which in the reproduction is a checkmark and an em dash sitting inside an assertion message [9]. And PYTHONUTF8 has to be unset, since the author credits the `PYTHONUTF8=1 LANG=C LC_ALL=C` run with making a clean server behave like his laptop, though the published sentence is cut off there [14]. Break any one of the four and the same command prints a readable failure [10].

The two candidate patches sit at different layers. That difference is the whole decision. The coding model the author consulted proposed stripping non-ASCII from assertion messages; he reports it would have silenced the crash and destroyed the signal [16]. It also leaves a writer that still cannot encode whatever the next contributor types. `PYTHONUTF8=1` in the runner environment fixes the writer and leaves the message alone [14]. I would take the environment variable, on the grounds that a reporter which aborts on its own output is the defect, not the typography.

What the piece does not establish is prevalence. It is explicitly a 48-hour lab notebook rather than telemetry from a fleet, and the author says to treat the commands as reproductions rather than as claims about your images [17]. The generalisation that many cloud images still boot with `LANG=C` or `LANG=POSIX` arrives without a named distribution, base image, or tag [4]. So the transferable part is the procedure, not the statistic: run `python diag_stdio.py`, then the same script under `LANG=C LC_ALL=C`, then again with `PYTHONUTF8=1` added [13].

That procedure is cheap. The dump prints nine pieces of state per run, including both stream encodings, `locale.getencoding()`, `LANG`, `LC_ALL`, `PYTHONIOENCODING` and `sys.flags.utf8_mode` [12], so three runs give 27 lines [18]. Only two comparisons matter: run one against run two isolates the locale, and run two against run three isolates UTF-8 mode. Everything else is there so you can prove the runner is not lying to you.

The reason this class of bug costs two days rather than ten minutes is the ordering [1]. The pinned interpreter and the pinned lock file make the two machines look identical [2], the failure arrives after a long install phase, and the artifact that would name the cause is the artifact the cause prevents from being written [3]. The message was formatted to be readable by humans, and that same formatting is what stopped the writer from printing it, so nobody read it.

What to watch

  • Whether the truncated third run gets published in full, showing what locale.getencoding() reports when PYTHONUTF8=1 is set under LC_ALL=C.
  • Whether anyone names a base image or CI runner that actually boots with LANG=C, since the post names none.
  • Whether pytest's reporter grows a fallback for characters stdout cannot encode instead of letting the write abort the run.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories