Build1 distinct publisher3 min readUpdated
A pruning demo cut a 20-trial study from 13.0 to 9.4 seconds while abandoning 12 trials. The distance between those two numbers is a ceiling the pruner settings impose.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
The author's explanation stops one step short of the arithmetic. Twenty trials in 13.0 seconds is 0.65 seconds per completed trial [1]. Had the 12 killed trials cost nothing, the eight survivors would have finished in about 5.2 seconds [2]. The pruned run took 9.4 [3], so the dead trials burned roughly 4.2 seconds between them, about 0.35 each, a bit over half the price of running one to the end [3]. Close to 45% of the faster run's wall clock went into trials whose results were discarded [4].
The settings explain where that floor sits. MedianPruner was built with n_startup_trials=5, so the first five trials could not be pruned at all [5], leaving 15 eligible. Twelve of those 15 died, which is 80% of everything the pruner was allowed to touch [5]. It killed nearly all of what it could reach and still returned 28% [3].
The other knob sets the rest. The model reports every 40 trees up to 200 [8], and n_warmup_steps=2 means two reports before a trial is eligible [5], so the earliest possible stop is at 80 trees, with 40% of the tree-fitting work already spent [6]. Assume per-tree cost is roughly flat, and the ceiling for this study is five full trials plus 15 partial ones at 40%, or 11 trial-equivalents out of 20: about 7.2 seconds, a 45% saving, and only if every eligible trial dies at its first legal opportunity [7]. The measured 28% is not far off what this configuration permits. You buy the difference back by lowering both knobs, and you pay for it in the risk the author prices at 0.0005 log loss [4], 0.26% of the baseline [9], on a search where he notes a trial that looks hopeless at 80 trees can be excellent at 200 [11].
The scaling argument in the post is that pruning is a wall-clock optimisation whose value tracks what a trial costs, so 13 seconds is noise and 13 hours is one overnight run instead of three [9]. The ratio, though, does not scale with the clock. Twenty-eight over sixty is 0.47 [8], and that fraction is fixed by the startup count, the warmup steps and where the bad configurations happen to fall, not by how long a single fit takes. A 13-hour study with this pruner and comparable luck saves 28%, and whoever sized the window from the kill rate is out by roughly half.
The production advice makes the floor heavier. The author says the pruning signal should come from cross-validation rather than the single validation split used here [10]. Cross-validation multiplies the cost of every report by the number of folds [10], and the reports you cannot avoid are precisely the warmup ones charged to every trial you later throw away, so a five-fold signal makes the unavoidable part of a pruned trial about five times more expensive in absolute terms. Pruning does not take the bad candidates off the bill. It issues a partial refund, and the size of the refund is something you set in the constructor [5].
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
Adding two lines inside the Optuna objective, trial.report(loss, step) and a trial.should_prune() check, let Optuna abandon 12 of 20 trials before they finished, which the author describes as 60% of the search.
The 20-trial study took 13.0 seconds with NopPruner and 9.4 seconds with MedianPruner, which the author reports as saving 28% of the wall clock.
Best log loss was 0.1908 with no pruner and 0.1913 with the median pruner, a cost of 0.0005 log loss.
The pruner was optuna.pruners.MedianPruner(n_startup_trials=5, n_warmup_steps=2); the author states n_warmup_steps=2 means two reports before any trial is eligible to be pruned, which is the floor on what can be saved.
The author's stated reason the two numbers do not match: a pruned trial is not free, because it runs its warm-up steps before there is enough evidence to kill it, so you pay the first fraction of every bad trial.
The author writes that pruning advertised as '60% fewer trials, 60% less time' is the advertisement, not the measurement.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Reproducible script, one unreplicated run
The post publishes a complete runnable script and its printed output, so the mechanism and the arithmetic are fully checkable, and the derived per-trial economics follow directly from the disclosed figures. But the evidence base is a single execution on one machine, one seed, one synthetic dataset and one validation split, with no repeats, no variance estimate and no second publisher to corroborate the timings — so the specific 28% figure is weakly grounded even though the structural point (warm-up work caps the saving) is arithmetically sound.
No adoption signal
The only observation is the author's own demo benchmark. The sources contain no deployment, usage disclosure, download, release or production-adoption evidence for Optuna pruning, and a self-run tutorial script is not adoption evidence.
Broadly aligned, mild generalisation from one run
The story is unusually self-deflating: it explicitly refuses to quote the kill rate as the speedup, discloses the log-loss cost, and says the saving is irrelevant at 13-second scale. That keeps claims close to evidence. The residual overstatement is scope: a promotional 'replace your grid search tonight' framing and an untested 13-hour extrapolation are extended from a single unreplicated toy configuration, and the post does not surface that its own warm-up settings cap the achievable saving near 45%.
Engagement and series growth, no disclosed vendor tie
The post is part of a self-promoted tutorial series and closes with explicit audience-growth calls to action ('Follow me for the next article', 'drop a heart', comment prompts), plus forward references to the author's other articles — an incentive to produce a punchy headline metric. Offsetting this: no vendor, sponsorship or commercial relationship with Optuna is claimed or evident, the tooling is open source, and the post volunteers its own negative findings.
Moderate: verifiable mechanism, thin measurement base
Confidence in the mechanism and in the direction of the finding is high because the code and the disclosed pruner settings make the reasoning auditable and the derived figures internally consistent. Confidence in the magnitudes is limited by a single publisher, a single execution, no repeats or seeds, a toy dataset, and no adoption or third-party corroboration.
build
Three manual interventions in a month, and every guard was working as designed1 distinct publisher
build
Six MariaDB versions, one real difference: the only reason to leave 10.6 is the July 2026 clock1 distinct publisher
build
Force the tool call, then hand Lightsail a long-lived key1 distinct publisher
build
AI-written code fails the same four ways, and every gate you own reports green1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 24, 2026