Build1 publisher2 min readPublished
Cache lookahead brings a kaniko fork within 1.6x of BuildKit on GitLab runners
Kaniko's community fork, with cross-stage cache lookahead, ran 1.6x slower than BuildKit on GitLab.com runners in a dev.to benchmark. The 15x gap still quoted against kaniko dates from 2018, so teams choosing an in-cluster builder need figures from their own Dockerfiles.
The Engineer · Build desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- The 2018 test ran kaniko without the --cache-copy-layers flag while BuildKit had a local disk cache, according to the author of the new benchmark.
- Older kaniko could not compute a COPY --from cache key before building the source stage, so it rebuilt every stage on every run just to find cache hits.
- Google's last official kaniko release, configured fairly by the author's account, trailed BuildKit by 7.5x in the same GitLab.com runner test.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision A team adopting kaniko has to choose between the community fork and Google's last release as well, since the two sit about 4.7x apart on the same runners.
- constraint The lookahead gain applies to multi-stage builds that use COPY --from, so teams with single-stage Dockerfiles should not assume the fork's 1.6x result carries over.
- cost Teams that pick kaniko for its daemonless design pay a startup cost and a tarball write per layer on every build. Short builds on slow disks pay the most.
A COPY --from line copies files out of an earlier stage. Old kaniko could not compute that line's cache key without first building the stage it copies from [3]. So every stage got built on every run, even when a cache hit was waiting at the end [4]. The fork's fix is good engineering. Lookahead works out the key across the stage boundary, and the author credits the gain to the redundant builds it removes [13].
The 2018 run added a configuration gap on top of that design limit [2]. A benchmark that hands one side a warm cache mostly measures the cache. According to the post, the missing --cache-copy-layers flag left kaniko rebuilding layers that BuildKit read from cache [12].
The new numbers come from GitLab.com runners, with both tools configured the way a CI pipeline would run them, the author says [5]. The post states its results as ratios. If BuildKit takes one minute, the fork takes about 96 seconds and Google's last official release takes seven and a half minutes [3]. On those figures the fork is about 4.7 times faster than the official release [2]. The official release's 7.5x is half the 15x from 2018 [1][1]. Two different tests produced those figures, so the halving cannot be credited to the flag alone [5].
For the 1.6x [6] to hold on another cluster, the builds would have to resemble the test. They would need to be multi-stage, with COPY --from lines, since that is where lookahead applies [3]. Disk speed would need to be comparable, because kaniko writes every layer out as a tarball while BuildKit streams layers directly [8]. Build times would need to be long enough that kaniko's startup stays a small share of the total. With no daemon holding a cache, kaniko pays that startup cost on every build [9].
The author calls the remaining gap deliberate, the price of putting security and portability ahead of speed [10]. The post describes the gap as "a context-dependent differential that must be evaluated based on workload requirements" [11]. I think that is the right frame. In the test scenario, the fork's builds ran about 60 percent longer than BuildKit's [4]. Whether that is a fair price for a builder with no daemon depends on the cluster. A team finds its own figure by running both tools on its slowest Dockerfiles, each with its best cache settings.
What to watch
- Whether the linked benchmark publishes wall-clock times and Dockerfiles so others can rerun the GitLab.com comparison.
- Whether cache lookahead reaches a kaniko release that teams still on Google's last official build can move to.
- Independent reruns on runners with slower disks, where kaniko's per-layer tarball writes should cost the most.