Skip to content

BuildNot yet confirmed elsewhere1 publisher3 min readPublished

OpenAI's Asana case study prices 1.5 weeks of engineer time against a five-year staffing plan

The two-week Enzyme removal is credible enough, but it is priced against a five-year staffing plan written by people who did not want the project, and Airbnb's published numbers let you check the rate that plan assumed.

The Engineer · Build desk

How we use AISend a correction

Illustration accompanying OpenAI's Asana case study prices 1.5 weeks of engineer time against a five-year staffing plan
Generated illustration

What happened

  • OpenAI's case study says Asana used Codex to remove Enzyme, an outdated testing system, in two weeks, with model and infrastructure costs of about $12,000.
  • The plan that figure is compared against was expected to take at least five years and to cost roughly $6 million, according to the same case study.
  • OpenAI puts the human input at 1.5 weeks of engineering effort spread across two calendar weeks, after which Enzyme was fully removed.
  • Airbnb migrated a 3,500-file Enzyme component test suite in six weeks using LLMs, against an estimate of 1.5 engineering years to do the same work by hand.
  • Gergely Orosz reckons the $6M came from assuming an engineer could migrate five to ten tests a day, then multiplying the resulting engineering years by salary.

Why it matters

  • cost The published $12,000 is a models-and-infrastructure number, so any per-migration budget copied from it quietly omits the engineer weeks and the human-finished tail the same class of project still needs.
  • constraint With one side denominated in budgeted engineer-years and the other in calendar time, the case study cannot support a productivity multiple; it can only show that an unloved plan was never going to be funded.
  • capability The transferable part of Airbnb's result is the retry loop and the purpose-built refactor pipeline, which means a team that buys the model without building the harness buys none of the throughput.
  • decision Anyone putting a vendor ROI figure in front of a board now has to name who wrote the counterfactual and whether they wanted the project, because that authorship sets the size of the saving.

Enzyme and React Testing Library are not two dialects of the same test. An Enzyme test operates on a component instance; RTL asserts against the rendered DOM, so the test sees the whole rendered page rather than the component it was written for [6]. Assertions have to be re-expressed in terms of what a user can see, and for complex user journeys the two approaches diverge sharply [7]. Asana went from the first to the second [4]. Both sides of the comparison are describing genuinely awkward work.

The units are where it comes apart. The arithmetic behind the $6M is four engineers at circa $300K a year for five years, according to Gergely Orosz's reading of the case study [5], and it multiplies out cleanly to the published figure [16]. That is also about 1,040 engineer-weeks of budgeted labour [17]. The other side of the comparison is 1.5 weeks of engineering effort [3], and the write-up does not say how many engineers those 1.5 weeks cover, which is a curious omission in a comparison this lopsided.

The $12K is model and infrastructure spend, not the cost of the work [2]. Charge the engineer time at the salary the $6M itself assumes and you add roughly $8,650, for about $20,700 all in [18]. Against an advertised gap of $5.99M [19], that is noise either way.

Airbnb's numbers let you test the rate the plan assumed. Its manual estimate was 1.5 engineering years for 3,500 Enzyme component test files [8]. At 250 working days a year, that implies about nine files per engineer-day [14], the top of the five-to-ten range Orosz guesses Asana's planners used [20]. Push Asana's implied 20 engineering years through Airbnb's rate and the plan is sized for roughly 46,700 test files [15]. If Asana's suite is nowhere near that, the five years measured appetite rather than throughput. Orosz puts it plainly: you estimate a project like this when it is work you really do not want to do [13].

Airbnb also shows where the effort actually sits. Its team built loops that retried failed migrations, and once those existed 75% of files went through in four hours [9]. The remaining quarter needed a purpose-built refactor pipeline, which took the total to 97% over four days, with engineers finishing the last 3% by hand over a week [10]. That was March 2025, on Claude 3.7 Sonnet [11]. The bulk was hours; the tail was a week of humans.

Orosz still finds two weeks credible for Asana, a year on from Airbnb [12]. The outcome can stand while the denominator gets audited. Whether a saving like this transfers depends on who authored the counterfactual and whether they wanted the work, the unit both figures are denominated in, and what the smaller figure leaves out. Calendar weeks set against engineer-years fails on the second before you get to the third.

What to watch

  • Whether OpenAI or Asana publishes the test file count and the per-engineer-day rate the five-year plan assumed.
  • Whether Asana's engineers describe a retry loop or refactor pipeline, since the case study credits the model rather than a harness.
  • Whether the next vendor case study quotes engineer-weeks on both sides of the comparison instead of calendar time on one.

Clarity's read

What the record supports and how the coverage leans. The claims behind it follow.

Reality

Evidence48
Adoption58
Hype gap+52
Incentives76
Confidence54
Why these scores

Claim ledger

Ranked by verification strength, evidence, and original report placement.

  1. [1]

    OpenAI's case study states the previous staffing plan for the work was expected to take at least five years and was estimated to cost roughly $6 million.

  2. [2]

    OpenAI's case study states that Asana used OpenAI Codex to remove Enzyme, an outdated testing system, in two weeks, with model and infrastructure costs totalling about $12,000.

    ReportedSupportedView cited source
  3. [3]

    Per OpenAI, Enzyme was fully removed after 1.5 weeks of engineering effort spread across two calendar weeks.

    ReportedSupportedView cited source

Sources

1 independent publisher whose own reporting we read for this story.

  1. blog.pragmaticengineer.com

    1 article · August 27, 2026

    The Pulse: We need to talk about migrations with AI

Share your take

Let Clarity write the post for you.

Signed-in readers get a short post drafted on this story in the register they choose — narrative, analytical, or a direct position — editable to the last word before it goes anywhere. The share buttons at the top of this story work without an account.

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Loading related stories