Science1 publisher2 min readPublished
Universities are redesigning assessment ahead of the evidence on AI and critical thinking
A Nature editorial says the studies on AI and critical thinking are still too early for firm conclusions, while MIT's own review of staff and student use warns that surviving the encounter will take substantial adaptation.
The Scientist · Science desk
What happened
- A Nature editorial, summarising its own News Feature on the literature about AI and critical thinking, says the studies are at an early stage and it is too soon to draw definitive conclusions.
- The same editorial says AI use is already widespread and that teachers are reporting signs that chatbots are reducing students' opportunities for thinking and reasoning.
- Last month MIT announced the results of a review of AI use by its staff and students, which the editorial draws on for both what students are doing and what teachers should keep doing.
- Universities are handing work previously done by humans to AI tools, with chatbots answering the administrative questions of first-week students and a few institutions experimenting with AI-generated lectures.
- Thomas Corbin, who studies assessment and digital learning at Deakin University in Melbourne, says students report that AI feedback does not have to be waited for and is available around the clock.
Compiled by The ScientistSomething wrong?How this is made
Why it matters
- decision Handing the marking over still leaves teachers tracking where each student is: Nature says teachers must stay up to speed with progress even when feedback goes to AI tools, and that there is no shortcut, so the staff saving is smaller than the invoice implies.
- constraint A university can verify how fast feedback arrives. The capacity the feedback is meant to build lacks an agreed measure, so two institutions could claim success for opposite redesigns and neither could be checked.
- contradiction Students name speed as the gain and teachers name lost reasoning practice as the cost. Both are self-reports, they point in opposite directions, and neither is an outcome measured on learning.
- precedent Tools are entering the teaching loop while the studies are still young. The burden of proof lands on whoever later wants to show the harm, and the institutions buying now carry none of it.
One reason the studies are slow to converge is that they are not all measuring the same thing. Researchers who study cognition describe critical thinking through the high-level processes underneath it: memory, reasoning, decision-making and problem-solving, each involving different but overlapping brain mechanisms [14]. Social scientists who study teaching and learning mean the ability to weigh up evidence, to consider trade-offs when deciding, and to take account of someone else's view in an argument [15], and an instrument built for the first set does not report on the second [1].
The timetable is moving regardless. Universities around the world are deep in preparation for what they expect to be a revolution in teaching, learning and assessment, and Nature puts the expected scale above the arrival of personal computers, e-mail and the Internet [4]. The task moving first is feedback on coursework submissions [10].
MIT's report ties the risk to scheduling, a variable a department can change inside a single term. "Not surprisingly, students told us the temptation was greatest when they feared they would miss a deadline," the report says [6]. It also says there will need to be "substantial adaptation to survive an encounter with AI without serious disruption" [7].
Both accounts of critical thinking need practice. Applying the techniques well takes deep subject knowledge and hours of work at tasks and exercises, including learning from mistakes, what teachers call teachable moments [16]. The editorial's firmest claim is about that loop: the feedback inside a student-teacher relationship, the light-bulb moment when a student recognises an achievement after a lot of effort, is irreplaceable [17]. That is the editorial's own judgement, offered without a supporting study.
Nature stops short of asking universities to slow the redesign down. What it says is essential during the transition, echoing the MIT report, is that the human teacher keeps the guiding role of showing students how to learn to think while they use AI [19].
The history is less comfortable for the worried side, and any study reporting harm now argues against that record. Calculators replaced logarithmic tables and word processors replaced essays written by hand or on typewriters, and the objection then was that a machine able to correct errors would stop students investing time in thinking about what to write; neither technology ended critical thinking [18].
Philippe Aghion, the Nobel-prizewinning economist, made the case for the human role earlier this month at a Nature conference in Paris on AI in health care: "We need schools in which we learn to learn" [13].
What to watch
- Whether any study in the literature Nature reviews reports a controlled comparison against a pre-specified measure of critical thinking.
- Whether universities that automate coursework feedback publish how instructors keep track of student progress once marking leaves their desks.
- Whether any department answers the MIT deadline finding by changing submission schedules, the cheapest variable it controls.