The person who answers for this on Friday is whoever owns the eval set, and in most teams the eval set is a folder of screenshots from the last time someone had a spare afternoon. That folder, not the scoreboard, is the real constraint here.
Start with the margins. Seven of the sixteen benchmarks went to a rival [7]. On DeepSWE-1.1, the long-horizon coding test, Gemini 3.8 Flash scored 73.7%, about a point ahead of GPT-5.6 Sol and a few fractions of a percent behind Claude Opus 5 [8]. On CyberGym, the cyber variant's lead is 2.4 points over Claude Mythos 5 and 2.6 points over GPT-5.6 Sol [12]. Those are gaps that a prompt rewrite or a change of harness can open or close inside your own stack, and all of them come from Google's own testing as reported by SiliconANGLE [6].
The mechanism is stated plainly in Google's post, which is more than most vendors manage. Tulsee Doshi and Raluca Ada Popa wrote that 3.8 Flash "works harder", executing extra reasoning steps and calling tools iteratively, and that at higher effort levels the model might use more tokens to maximise performance [10]. So the figure to log next to accuracy is tokens per completed task on your own traffic, not the rate card.
The two models also have two different audiences. General-purpose Flash is for anyone with an API key. The cyber variant is for the Fairwind list: government agencies, critical infrastructure operators, and firms Google describes as securing widespread software foundations [4]. If your name is not on it, the security-tuned model stays something you can read about in a press release, not something you can run yourself.
Teams describe a tidy process for when a release lands: re-run the suite against the new model, compare cost per task, then decide. What tends to happen instead is that someone edits the model identifier in a config file, nothing throws an error, and the quiet gets recorded as a pass. Quiet only measures whether your failures are loud, not whether the output got better.
A sort that survives a three-week release cadence [1] needs two axes. First, do you have a scored set of your own tasks that one person can re-run in an afternoon? Second, does a two-point accuracy move change something that ships, or only a draft a human reads before it goes out? Workflows with both a scored set and a shipping consequence are worth re-testing every time the version number moves. Workflows with a human in front of the output can ride a release or two behind without anyone getting hurt. The workflows with a shipping consequence and no scored set are the actual backlog item, because until that set exists you will keep making this call from a table somebody else ran.
