Human-vetted AI lesson plans save Karnataka teachers two hours a week, by their own account

About 96.1% of the English plans went through review with no adjustment at all ^[18]. That is nearly the inverse of the Kannada result ^[15]. Roughly 26 of the 7,744 English plans were rejected outright ^[19]. Most of the Kannada edits were about terminology: "mirrors" in place of "glasses", or "timeline" in place of "chronology" ^[15].

The researchers put the human review at the centre of the design. "With English, there are lots of resources available on the internet already, but that's not the case for languages like Kannada," said Deepak Varuvel Dennison, a doctoral student in information science and the paper's first author ^[2]. "Here, we have a tool where the lesson plans have already gone through a vetting process by experienced teachers who ensure they are high quality and devoid of any errors," he said ^[3].

Tanuja Ganu, director of research engineering at Microsoft Research India, said the plans had to be "well-tailored, in terms of the cultural nuances, local languages, local curriculum, and the resources which are available." "That was, if not more effort, at least equal effort to building the technology," Ganu said ^[4].

The classroom claims come from a different kind of evidence than the review counts. The time saved, and the relief from chores such as submitting hard copies of lesson plans, were reported by teachers in interviews and surveys ^[5]. The phys.org account does not describe a comparison group or any measure of what students learned. The system also bundles three things: the models, the curators, and each teacher's own customization ^[12]. The reported results cannot separate which of them produced the gains.

I would still take the teacher reports seriously as evidence about adoption. Tools that add work tend to be dropped. "Usually, whenever someone comes with something new, it just adds to our workload, but this is a burden-releasing concept," said a science teacher with 20 years of experience ^[6]. The hands-on examples teachers gave were specific, such as collecting local leaves with different vein patterns ^[7]. That is a credible sign of changed lesson plans. It is weak evidence of changed learning.

The evaluation was run with the Sikshana Foundation, a local nonprofit that works to improve government schools ^[13]. Aditya Vashistha, an assistant professor of information science at Cornell, framed the project around users that technology was not built for. "Technologies are designed for the first 1 billion people who use them, and an enormous amount of effort is required to take the technology to the next billion, and the next billion, and so on," he said ^[8].

Varuvel Dennison will present the work at the ACM Computer-Supported Cooperative Work and Social Computing conference in Salt Lake City, held Oct. 10 to 14 ^[11]. The paper received an honorable mention ^[11]. The team is now testing Shiksha Copilot with a larger group of teachers ^[17].