Build1 publisher3 min readPublished
Under Docker 29's default image store, a 100MB container quota let 300MB through
On an ext4 host the overlay2 graphdriver refused --storage-opt size outright with exit code 125. The containerd snapshotter that Docker 29 makes the default accepts the same flag on the same filesystem and enforces nothing.
The Engineer · Build desk
What happened
- Docker Engine 29 makes the containerd snapshotter the default image store on new installs, in place of the classic overlay2 graphdriver.
- On a fresh 29.3.1 install with an ext4 root and no project quotas, the graphdriver rejected docker run --storage-opt size=100M with exit code 125 on three consecutive runs, and no container started.
- With the containerd snapshotter active on that same host, the identical command ran, and a dd inside the container wrote a 300MB file to its writable layer and exited 0, again across three runs.
- docker inspect on the snapshotter run still returned map[size:100M] for HostConfig.StorageOpt, recording the option exactly as it would on a host that enforced it.
- In the same test sequence, pulling node:22 averaged 18.0 seconds on the snapshotter against 28.5 seconds on the graphdriver with max-concurrent-downloads set to 1.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure Any compliance script or audit tool that confirms a quota by reading container config now certifies a cap the kernel is not applying, so the first sign of a runaway writable layer is a shared disk filling up.
- decision Anyone who relies on writable-layer caps has to prove them with a write past the limit before putting 29.x on hosts, because the daemon's exit-125 refusal no longer arrives to flag an unsupported filesystem.
- constraint The result comes from a host that could never have honoured the option, so it does not support the broader claim that the snapshotter drops quota enforcement on XFS mounted with pquota.
- contradiction Docker's notes describe a snapshotter concurrency bug fixed in 29.7.0, and the 29.3.1 measurements show no serialization at all; on this host the unpack step, not the download count, set pull time.
The precondition on `--storage-opt size` is old and documented: it caps a container's writable layer only on overlay2 over XFS mounted with the `pquota` option [3]. The host in this test does not qualify, because its root filesystem is plain ext4 with no project quota mount option [2]. The graphdriver checks for that and stops the run with `--storage-opt is supported only for overlay over xfs with 'pquota' mount option` [4].
The backend is one daemon flag. Setting `"features": {"containerd-snapshotter": false}` selects the old path, and `docker info` then reports `Storage Driver: overlay2` instead of `overlayfs` with `driver-type io.containerd.snapshotter.v1` [10]. On the snapshotter side the same command is accepted, and a `dd` inside the container copied 314572800 bytes in 1.81 seconds [6]. Three times the stated cap, with no warning and no log line [1][7]. The author of the dev.to write-up attributes this to an unimplemented check: the containerd image store path does not validate that the backing filesystem supports a quota before accepting the option, where the graphdriver path does [9].
The speed result carries more conditions than the quota result. One host, one network path, one image: `node:22`, 8 layers, 1.64GB unpacked, pulled from a clean state with the local image removed each time and the backends alternated [11]. The graphdriver runs came later in the sequence, so a warm registry or proxy cache would have favoured the slower side [13]. With `max-concurrent-downloads` at 1, the gap is 10.5 seconds, about 37% [2]. Raising the limit to 8 moved the snapshotter to 15.0s and the graphdriver to 27.5s, widening the gap to about 45% [12][3].
Inside either backend, the concurrency setting barely moves. Going from 8 to 1 cost the snapshotter about 17% and the graphdriver about 3.5% [12]. Docker's own release notes record a bug, fixed in 29.7.0, where concurrent pull limits were not honoured on the snapshotter path, and 29.3.1 should have shown that as serialization [14]. The numbers show no such serialization. The author's conclusion from them is that unpacking layers into the filesystem dominates pull wall-clock time far more than the number of concurrent HTTP downloads does, at least on that network path [15]. For the 37% to transfer, your pulls have to be unpack-bound too.
Two limits on the quota finding are worth stating. The host was ext4, so the test does not cover whether the snapshotter honours `size` on XFS with `pquota`. And the daemon was 29.3.1, behind the 29.7.0 release that Docker's notes credit with graduating `--mount type=image` out of experimental; on 29.3.1 that mount type still prints `WARNING: Image mount is an experimental feature` [16][2]. The check that would settle either question on your own hosts is a write past the limit, inside a container, on the filesystem you actually run.
What to watch
- Whether Docker adds the filesystem-support validation to the containerd image store path, or documents --storage-opt size as graphdriver-only.
- A repeat of the same quota test on XFS mounted with pquota, and on the 29.7.0 line, which would show whether the option is enforced anywhere on the snapshotter path.
- Whether a later 29.x release note records the silent acceptance of --storage-opt size as a fixed bug.