Engagement wins as a metric because it is the metric the software produces for nothing. A learning platform logs session length and reward streaks as a byproduct of running; establishing that a 12-year-old can use a concept later, on paper, in a setting the vendor did not build, requires someone to design an assessment, administer it away from the product, and find students to compare against. That asymmetry is why dashboards became the evidence base for purchasing. The APA report declines to accept the cheap measure, arguing that applying knowledge outside the app is the much stronger indication that learning happened, and that frequent clicking or responding to animations does not necessarily mean it did [3][5].
Generative AI makes the gap wider by acting on the observable side of it. "AI can help a student produce a better essay, but the improved final product does not necessarily mean the student has become a better writer," said Nicole Barnes, the APA's executive lead psychologist for education, who added that schools will increasingly need to separate performance from learning as well as engagement from learning [9]. The report frames this as a construct validity problem, not a cheating one. The artifact a teacher grades and the capability a school is trying to build have come apart, and the report's stated risk is precisely that immediate performance rises without underlying knowledge or skill following [2].
Rather than call for exclusion, the panel recommends teaching students to interrogate what a chatbot returns: question its assumptions, look for gaps in its reasoning, compare it against other sources, and ask how the answer would change under different circumstances [10].
The procurement half is where this bites. The report places primary responsibility for evidence on the companies that build these products and on the institutions that adopt them, tells developers to be ready to demonstrate that their products improve learning, and tells schools and districts to expect credible, independent evidence before they spend [11]. APA chief executive Arthur C. Evans Jr. framed the recommendations as a way to determine whether tools actually produce the outcomes they promise [13].
What the summary leaves undefined is where that bar sits. The published summary sets out 10 recommendations without effect sizes, and without defining what makes evidence credible or independent [7][15]. Independent of whom, funded by whom, with transfer measured how long after use, against which comparison group: a district writing that phrase into a contract has to supply those numbers itself, and the guidance covers roughly 13 school years, ages 5 to 18, in which the right assessment looks different every couple of grades [4][14].
For parents the version is cruder and probably more usable. Spend a minute watching the screen; if the experience is largely dressing a character, collecting rewards or responding to flashing images, question the educational value, then ask a teacher or school board whether independent evidence exists that the app leads to learning [12]. Note the report's own hedge, which is doing real work: engagement does not *necessarily* mean learning [5]. The panel treats attention as a precondition it never dismisses, not as the outcome it scores.
