Science2 publishers2 min readPublished
A weekly 20-minute phone tutorial raised math scores in all five countries tested
Five randomized trials run during pandemic school closures found that a weekly 20-minute phone tutorial produced basic math gains in every country tested, while text messages alone gave mixed results across the same settings.
The Scientist · Science desk
What happened
- A Nature paper reports five randomized trials of remote tutoring during school closures, run in India, Kenya, Nepal, the Philippines and Uganda.
- More than 16,000 households with primary school students took part between December 2020 and July 2022.
- Students were randomly assigned to no remote tutoring, to weekly text messages with math problems and practice reminders, or to those texts plus a weekly 20-minute one-to-one phone tutorial.
- In the phone group the number of students correctly solving division problems rose 92%, and performance on fractions improved by 6.4 percentage points.
- phys.org places the combined program, at $11 per child, among the top 10 most cost-effective education interventions of 150 evaluated worldwide.
Compiled by The ScientistSomething wrong?How this is made
Why it matters
- capability The effect held whether government teachers or NGO instructors made the calls, so a ministry can mount this on staff it already employs and a list of phone numbers, with no platform to procure or connect.
- constraint An internet-delivered closure plan can reach under 15% of households in low-income countries, against more than 70% reachable by phone. That gap caps how much of a population any app-based continuity plan can serve.
- contradiction The two write-ups price the same program differently, at 3.6 years of schooling per $100 and at approximately 4, so a budget built on the higher number claims about a tenth more value than the detailed account supports.
Each call opened with a short assessment. The instructor used it to work out what the child could already do, then built that week's lesson on it instead of following the grade-level curriculum [6]. Twenty minutes a week for eight to twelve weeks comes to between 2 hours 40 minutes and 4 hours of one-to-one instruction per child over the whole program [1]. Divide the per-child cost by that time and the combined package ran at roughly $2.75 to $4.15 per hour of live tutoring, text messages included [2].
Cost-effectiveness here is a conversion. The metric turns test-score gains into an equivalent quantity of schooling, and on the lower of the two published accountings, $11 per child works out to about 0.4 years, close to five months [3]. No child in these trials sat through five months of instruction.
phys.org writes that students learned roughly three times more when they received phone calls tailored to their learning level [23], and further down gives the comparison: the average gain of 0.321 standard deviations was three times larger than the median education intervention evaluated globally [7] [8]. The reference point for that number, then, is the global median. On the text messages, the authors wrote in Nature: "Whereas text messages have mixed results, tutorials conducted via phone call show effectiveness across diverse settings" [15]. Neither source reports how much the calls added on top of the texts, and the SMS-only arm was not run in India at all [4].
The effect size also does not tell you how these children compared with children sitting in classrooms. Schooling was disrupted by COVID-19 in all five settings, and several of the study countries had some of the longest school closures in the world [24]. The gain is measured against children who received nothing remote while school was unavailable. That is the comparison an emergency plan actually faces, and the only one these trials could run.
Outcomes were basic math, tested at endline with the Annual Status of Education Report tool [19]. Neither source reports a reading result, and neither reports a follow-up test after the tutoring stopped. Education Cannot Wait, the UN fund for education in emergencies, estimates that 222 million children experience regular schooling disruptions [18]. Uganda and the Philippines saw the biggest improvements [20].
What to watch
- A follow-up test months after the calls stop, which would show whether the basic math gains hold.
- A costed national rollout by one of the study governments, with teachers carrying full class loads, to test the $11 per child outside a trial.
- A reading outcome from the same design; these five trials report basic math only.