Loading today’s stories
Build1 distinct publisher2 min readPublished
Three seeding habits produce three predictable flake signatures. The fix in a dev.to walkthrough is less about Playwright than about what determinism does to the assertions you are allowed to write.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
Start with the arithmetic. The dev.to walkthrough's example setup is four UI steps: sign up, log in, add a product to the cart, check out [2]. If fifty specs each need that order to exist, the suite drives 200 UI flows before reaching an assertion anyone wrote on purpose [12]. The second cost is worse than the minutes. Setup runs through the interface under test, so signup becomes a dependency of every spec that needs a logged-in user, and one regression there reports as fifty failures wearing fifty different symptoms [2].
Fixture SQL fails in the least useful way available. The insert error points at a row, not at the renamed column or new required field that invalidated forty of them [4]. Data regenerated from a definition per run at least fails at the definition.
The batch payload is where the design earns its keep. One entry asks for `tpl_user` with alias `user` and count 1; the second asks for `tpl_order` with count 3 and a relation declaring that `userId` comes from `user` by round-robin [6]. Four records come back from two entries [11], and the three orders already carry the generated user's id, so nothing has to patch `userId` afterwards [6]. That is the difference between a relational generator and a pile of independent factories.
The config is honest about ownership. `resetTestDatabase()` is your own truncate code, called from `globalSetup` before the fetch [5]. The generator itself is an HTTPS POST to `api.jsonfabrica.com/v1/batches` carrying a bearer key from the environment [8]. That is cheap at ten rows and less cheap at scale, because a large batch returns a `batchId` you poll instead of records you can write straight to a fixture file [9].
The example seed is 20260903, which looks like the day someone wrote the post. Determinism is indifferent to taste in magic numbers [6].
Scope is handled properly: if you are intercepting network calls so a frontend can run with no backend at all, this is the wrong tool, and E2E data differs from a developer's local database in that it is short-lived, reset per run or per CI job, and read by assertions that check exact values [10]. That line is where I would hold too. In my context the backend is the component most likely to be wrong, so I want real rows and exact expectations, and I will pay a network hop in `globalSetup` to get them. If your risk sits in how the frontend renders arbitrary payloads, mocked responses buy more per engineering hour, and the fixed seed buys you nothing you can assert on.
Ranked by verification strength, evidence, and original report placement.
The prescribed fix: generate the data the suite needs from a template definition with a fixed seed, right before the suite runs, and load it directly rather than through the UI. The batch API is called from globalSetup (wired via globalSetup: require.resolve('./e2e/global-setup.ts') in playwright.config.ts) before any spec file runs, after the team's own resetTestDatabase() truncate/reset code, and the response is saved to a fixture file specs import.
Playwright test data is defined as the database rows or API records an application must already contain before a browser test runs (a logged-in user, their orders, the products those orders reference), generated deterministically so the same run produces the same data every time; unlike a unit test, a Playwright or Cypress spec drives a real browser against a real running app, so the backend needs real rows rather than an intercepted network response.
Pattern one: tests create their own data through the UI. A test needing an order first signs up a user, logs in, adds a product to a cart and checks out before the assertion it cares about. That is slow multiplied across every spec needing similar setup, and the thing under test is also the thing doing setup, so a bug in signup breaks fifty unrelated tests.
Pattern two: a shared, mutable test database. If every spec reads and writes the same rows, test order starts to matter; a test that deletes a user breaks a later test that assumed the user still exists. The post calls this one of the most common sources of a suite that passes locally one file at a time and fails intermittently in CI when specs run in parallel or in a different order.
Pattern three: hand-maintained fixture SQL or JSON. A fixtures.sql file or a static users.json works until the schema changes; a column is renamed or a new required field is added, and the fixture silently stops matching what the app expects, or starts failing inserts with no clear signal about which of forty rows is the problem.
The example payload uses seed 20260903 with two document entries: templateId tpl_user, alias 'user', count 1, params email [email protected]; and templateId tpl_order, alias 'order', count 3, relations userId from 'user' with strategy round-robin. Posted to the batch generation API it returns one user and three orders where every order.userId is that user's generated id, avoiding separately generated orders whose userId must be patched in afterward.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · September 3, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
CI cannot tell a regression from a stale test because nobody wrote the intent down1 distinct publisher
build
The repo's own control run deleted the 5-10x WASM claim from vizcrush's launch copy1 distinct publisher
build
A Passing AI-Generated Test Is Not Evidence: Seven Checks Before It Enters Your Suite1 distinct publisher
build
Five rewrites later, the LLM is out of the test loop and into the selectors1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Runnable mechanics, one voice
Most of this you can verify by pasting it into a repo and watching it run: the seed-and-templates request, the config wiring, the shape of the results array, the assertion that counts seeded orders. What nobody has checked is the premise — that these three habits produce those three flake signatures, and that swapping in a generated batch cures them. dev.to shows the code and asserts the outcome; the outcome is where the sourcing stops.
No usage on record
Not one team, suite size, CI timing, or call volume appears anywhere in this reporting. A worked example is not a deployment, and we decline to read one into it.
Restrained pitch, unpriced promise
Give the piece credit for the paragraph most vendor-adjacent tutorials omit: it says outright that the service inserts nothing and is not a Playwright plugin, and it distinguishes its problem from mocking and local dev seeding. The overshoot is narrower. 'Fast and non-flaky instead of slow and order-dependent' is doing a lot of work for a change that only fixes drift between runs, and every line of insertion, reset and isolation code stays with the reader.
The remedy ships with a bearer token
The three diagnoses are vendor-neutral; the cure is not. It routes through api.jsonfabrica.com with a JSONFABRICA_API_KEY, and the sync-versus-poll detail about seeding thousands of rows reads as capability marketing for that endpoint. Nothing dishonest — the seed-and-template idea works with any generator — but the piece never mentions one, and the interest in you calling this particular one is plain on the page.
Sure about how, silent about whether
Our confidence splits cleanly. On mechanics — which records that seed-and-templates request asks for, what globalSetup does with the response, what the endpoint expects in return — we are close to certain, because it is all on the page in code. On whether any of it makes a suite measurably faster or steadier, we have one author's word and no second reading, which is why the number sits mid-range rather than high.