Skip to content

Build1 publisher3 min readPublished

Duolingo builds a dedicated team to teach engineers AI use, with code review agents as a planned case study

In an InfoQ talk, Duolingo engineer Sarah Deitke describes the hands-on labs and usage dashboards her DevEx AI team built to make engineers comfortable with AI, and says they trust in-house content over a vendor's.

The Engineer · Build desk

Photograph accompanying Duolingo builds a dedicated team to teach engineers AI use, with code review agents as a planned case study
Photo: infoq.com

What happened

  • Sarah Deitke, a software engineer at Duolingo, works on DevEx AI, a team aimed at getting engineers to use AI effectively.
  • Her InfoQ talk sets out the internal AI literacy programming she says enables autonomous systems and agents, with a case study on redesigning Duolingo's code review process.
  • The program's first component is structured lab-style workshops: hands-on outlines that engineers work through at their own pace, written and vetted inside the engineering org.
  • AI observability dashboards that began as a daily active user chart now break consumption down by function, developer community, IDE, language, token usage, cost and model family.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • cost If the trust effect really comes from internal authorship, the expense is a staffed team writing and maintaining labs, and a vendor course bought at a lower price does not substitute for it.
  • constraint An org copying the dashboards gets a curriculum steered by tool consumption, and still needs a separate instrument to tell whether engineers accept what an agent reviewer says.
  • decision Before an agent gets review authority, someone has to name the accountable human, and Deitke presents that as an open conversation inside her team.
  • precedent Deitke's ask to the audience turns an enablement team into a budget line engineers can request from their own leadership, separate from the existing DevEx headcount.

A lab-style workshop is an outline. Deitke described hands-on materials engineers work through at their own pace, covering MCP servers, Cursor rules, batching requests to LLMs, and evaluations [5]. None of the four topics she showed is code review itself [14].

The usage problem and the acceptance problem are not the same. Teaching an engineer to write a Cursor rule teaches them what the tool does, and leaves open who signs off when the review comment comes from an agent. Deitke said her team has that conversation a lot: "With something like code review, maybe we saw both the engineer and the reviewer are responsible. When one of those is AI, who's responsible?" [11] She also said engineers can be skeptical, and that core systems like code review "are viewed as sacred" [10].

Then the authorship claim, which is the part that costs money. "engineers trust the pre-vetted content from their engineering orgs a lot more than, say, bringing in a vendor and doing a more generic solution," Deitke said [6]. If that holds, no vendor can supply the labs: someone on payroll writes and vets them.

The 95% was shown at a company all-hands last year, and what it reports is that engineers learned something new and would recommend the format [7]. For the number to transfer, two conditions have to hold in your org: the tools are already deployed, and the labs are written by engineers your audience already works with. Deitke asked the room how many of their organizations had Cursor or Claude Code, then how many had a team dedicated to helping people use them effectively [4]. Nothing in the survey measures whether an engineer merged an agent's review.

The dashboards are the most portable piece here. Deitke said the team built AI observability dashboards partly to work out what to put in the workshops, starting with a daily active user chart for one or two vendors and in-house tools [8]. Token usage is easier to chart than review quality. The dimensions she listed record consumption [9]. A curriculum chosen from them follows the tools engineers already open.

The transcript InfoQ published breaks off mid-sentence in the observability section, before the code review redesign Deitke outlined at the top of the talk [13][3]. On the record as published, Duolingo funded a team, ran internal training, and measured how the training landed with engineers. The step from those workshops to an autonomous reviewer is in the talk's outline. The transcript carries training numbers and no review numbers.

I would still copy the dashboard-to-curriculum loop, because it is cheap and it tells you which developer community is actually using what. I would not expect the trust effect to arrive with it. Trust, in Deitke's framing, is slow to build and quick to lose [12], and the thing she credits for building it is content her own colleagues wrote.

What to watch

  • Whether Duolingo publishes the code review case study with acceptance or revert rates for agent-authored reviews.
  • Whether the observability dashboards gain a review-outcome dimension alongside token usage, cost and model family.
  • Whether any other engineering org reports the same preference for internally authored labs over vendor-delivered training.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories