Build1 distinct publisher3 min readUpdated
Seven failures in twelve runs of one Vesuvius Challenge test exposed three PermissionError surfaces in a chunk cache, and one of them under-evicts on every operating system.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
Zarr commits each cache entry by writing a temporary file and calling os.replace [11]. On POSIX, rename(2) over a file another process holds open is legal; on Windows, MoveFileEx with MOVEFILE_REPLACE_EXISTING returns ERROR_ACCESS_DENIED [12]. Content addressing turns that from an occasional collision into the steady state, because four workers reading the same volume derive the same keys and try to commit the same entries [13]. Repair the write path and the reader's open gets denied by somebody else's in-flight commit instead [15].
The third surface is the one that travels. The package's own LRU sweep caught only FileNotFoundError when deleting old entries, Windows raises PermissionError for a file still in use, and the sweep debited the file's bytes from the size budget anyway [16]. A cache whose entire job is staying under a byte ceiling was therefore under-evicting while its accounting insisted it had freed space, and that arithmetic is wrong on Linux too [16].
The rates deserve more attention than the headline percentage. Seven failures in twelve runs is 58.3% [6][7], which means a single-run CI job comes back green roughly 42% of the time [8]. That is the figure that explains how this lived in a hot-path component [9] without anyone filing it. After the fix, twelve consecutive runs were clean [20]. Treating runs as independent at the original rate, twelve greens in a row would turn up about once in 37,000 attempts [27], which is what makes that a verification rather than a lucky afternoon.
The remedy declines to branch on platform at all [17]. Refused writes become no-ops logged at debug, unreadable entries report as a cache miss so the outer cache refetches from source [17], and the sweep now skips undeletable entries without crediting their bytes and evicts the next-oldest [18]. The bill for that is stated rather than hidden: a full disk produces slow refetches instead of a crash [19]. The suite arithmetic is also tidy, 47 passed plus 2 failed is 49 before, 52 after, so the three new deterministic tests are the whole delta [22].
Two provenance notes. This is one self-reported account written for a contest [26], and the fix is a pull request against villa [24], not a landed upstream repair. The upstream zarr issue has been open since October with three confirmations and no fix [14], so any downstream project touching that store owns the rename semantics itself for now.
There is also a reason nobody had run this suite off Linux: uv sync --extra all is unresolvable on Windows and macOS because the CUDA-only cucim-cu13 dependency ships manylinux wheels only [25]. The same trip turned up an LRU test that stamped files 1 to 3 nanoseconds apart, which NTFS at 100 ns resolution collapses to st_mtime_ns == 0, destroying the ordering the test asserts; ext4 resolves to 1 ns, so Linux CI never noticed [23].
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
The author set up the vesuvius package on a Windows 11 machine, working with an AI coding assistant to run the test suite on a platform it had never been tested on; one test failed, then passed, then failed again.
The test test_shared_cache_multiprocess_reads_are_not_torn spawns four processes that read one scroll volume through a shared on-disk chunk cache.
Run twelve times on Windows, the test produced 7 failures out of 12, all PermissionError: [WinError 5] Access is denied.
The cache is on the hot path for real usage: it is the component behind the package's documented volume_cache_dir config and the --cache-dir flag of its inference CLI.
According to the writeup, any PyTorch DataLoader with num_workers > 0 puts multiple processes into exactly this concurrent pattern, so on Windows training runs would randomly die mid-epoch.
The commit-path behaviour is upstream issue zarr-developers/zarr-python#3522, open since October with three confirmations and no fix.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Specific and repeatable in method, but single-source and self-reported
The account is unusually concrete for a single article: named test, exact Win32 error, a twelve-run failure count, the zarr commit pattern and POSIX/Windows rename divergence, a filesystem timestamp-resolution explanation for the CI blind spot, three monkeypatched regression tests verified against unfixed code, and a public PR with the three files touched. Against that, every figure comes from one contributor's machine and one publisher; no maintainer, upstream, or third-party reproduction appears in the cluster, and the diagnosis was AI-assisted with verification self-reported.
PR-stage; no merge, release, or upstream fix confirmed
The only adoption artifacts are a submitted pull request against ScrollPrize/villa and the author's local test-suite runs. Nothing in the cluster shows maintainer review, merge, a package release carrying the fix, or other users hitting or applying it. The upstream root cause in zarr-python is reported still open with no fix, and the project's own platform support remains Linux-only, so the fix's reach beyond one contributor's machine is unestablished.
Headline matched by measurement; impact scope runs slightly ahead of proof
The central numeric claim — 58% failure before, 0 of 12 after — is stated with its method and is modest rather than inflated, and the author explicitly downgrades one finding to a documented trade-off. The mild overstatement is in reach: 'training runs would randomly die mid-epoch' for any DataLoader with workers is inference from one test on one machine with no reported user incident, the fix is presented as done while the PR's fate and the upstream zarr fix are unresolved, and a contest framing rewards a clean before/after narrative.
Contest submission and personal portfolio incentive, partly offset by disclosure
The piece is explicitly a submission to DEV's Summer Bug Smash powered by Sentry, so there is a competitive and reputational reward for a dramatic, tidy bug-fix narrative, and a first-person contribution writeup also serves the author's portfolio. Offsetting factors: the sponsor's product is not promoted or benchmarked, the AI-assistant workflow is disclosed, the trade-off introduced by the fix is stated, and the underlying PR and upstream issue are linked for checking.
Mechanism credible and internally consistent; outcome and reach unverified
Confidence is moderate: the syscall-level explanation is coherent, corroborated in kind by an upstream issue with three independent confirmations, and the arithmetic in the derived claims checks out against the reported counts. It is capped by a one-publisher, one-author cluster with no independent reproduction, no merge or release confirmation, and an impact story that generalizes from a single environment.
build
Short Build Roots Do Not Fix MAX_PATH: Unreal Cook Failures Are a Windows Setting1 distinct publisher
build
A GPU SQL Engine Lost to One CPU Thread Because a Dispatcher Constant Was 128x Too Small1 distinct publisher
build
1,500 submissions in 14 days: what a 12th-place GPU kernel says about agent loops1 distinct publisher
build
Invoked in three runs, executed in none: the cost rule that never got asked1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 21, 2026