Skip to content

Build1 publisher3 min readPublished

Two steps consume 187 of the 198 seconds in Playwright's default CI run

A demo repo with about 40 Playwright tests spent 42 seconds downloading three browsers and 2m 25s running them one worker at a time. Checkout, Node setup and npm ci took the other 11 seconds.

The Engineer · Build desk

What happened

  • A dev.to post, first published on the Endform blog, lays out a Playwright and GitHub Actions setup intended to stay under five minutes on a single runner without sharding.
  • On a public repo with about 40 lightweight tests, the workflow generated by npm init playwright@latest ran in 3m 18s.
  • The rewritten workflow caches ~/.cache/ms-playwright under a key built from the runner OS and a hash of package-lock.json, and runs Chromium alone on pull requests.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint A cache hit still installs the OS libraries the browsers need, so the time recovered is less than the 42 seconds the install step originally cost, and nothing in the post separates the two.
  • contradiction The demo suite ran in 3m 18s before either edit, so the run offered as proof of the five-minute claim never had a five-minute problem, and no measurement here covers a suite long enough to tempt sharding.
  • decision Running one project on PRs instead of three trades cross-browser coverage for feedback speed, and a team that takes the trade finds its Firefox and WebKit failures on the merge to main.

Start with where the 198 seconds went. The browser install step took 42 seconds and the test step took 2m 25s [5][6], which is 187 of the 198 seconds in the baseline run [21]. Checkout, Node setup and `npm ci` came in at a second or two each and account for the remaining 11 [8][22]. Two config edits therefore sit on top of about 94 percent of the measured time in that repo [21].

Whether the split transfers depends on the shape of your suite. The download is a fixed cost: Chromium, Firefox and WebKit plus their system libraries, on every run, whether or not the browsers changed [5]. On a suite that takes 20 minutes, that same 42 seconds is roughly 3.5 percent of wall clock [24], and worker count is the only setting left that changes wall clock much.

Playwright puts its binaries in `~/.cache/ms-playwright`, which on the Linux runner resolves to `/home/runner/.cache/ms-playwright`, and that directory is what `actions/cache@v6` restores [14][15]. The key is `runner.os` plus a hash of `package-lock.json`, which the post calls a proxy for the Playwright version [15]. Any change to the lockfile produces a new key, so a bump to an unrelated dependency buys you a fresh browser download [27]. On a hit the workflow still runs `npx playwright install-deps`, because the restored binaries need system libraries that were never in the cache [16]. The original 42 seconds covered the download and those libraries together, so the recovered time is smaller than 42 seconds; the published excerpt does not report a measured time for the fixed run [5][30].

The generated config pins CI to one worker with `workers: process.env.CI ? 1 : undefined`, so the suite runs close to one test at a time even with idle cores [7]. The fix is one line. The recommended value is `"50%"`, scaled to the runner, with a comment in the config telling you to measure and raise from there [9]. `fullyParallel: true` belongs next to it, since it parallelises tests within a file and not only across files [10]. Under one worker, `fullyParallel` does nothing for wall clock [23].

Two other defaults push against the five-minute target. `retries: 2` in CI lets a flaky test run three times inside the same budget [11][28], and `trace: "on-first-retry"` means the attempt that first fails produces no trace to look at [12][29]. `forbidOnly` I would keep whatever it costs: with CI set, a committed `test.only` fails the build instead of passing a suite of one [13].

The five-minute promise deserves a check against the baseline. 3m 18s was already under five minutes before either edit landed [3][26], and the post says to treat that figure as "a fixed number for comparing fixes, not a prediction of your own run times" [4].

What to watch

  • A measured run time for the same 40-test repo after both edits, which would show how much of the 187 seconds the two config changes actually recover.
  • How the original 42 seconds splits between downloading binaries and installing the OS libraries, since only the first half is cacheable.
  • Whether raising workers above 50% on the same runner keeps helping or starts thrashing. The config comment tells you to test it.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories