Skip to content

AI tutors and tutoring

Part of the AI coach reading library: 30 papers cited in the AI coach notes, each in plain language. Who was studied, how, what it found, why it matters for our students, and where our notes use it. LLM tutors in schools, how students use general AI, tutoring software, and what tutoring delivers at scale. Download links open the archived copy and need an Avanti Google login; the Source link is the public original.

The overviews are drawn from the extraction tables and literature reviews written when each paper was read, not from a fresh reading. Where a number here and in a note differ, the note’s section is the one to check.

Generative AI Without Guardrails Can Harm Learning. Does a ChatGPT-style tutor help or hurt on an exam taken without it?

Section titled “Generative AI Without Guardrails Can Harm Learning. Does a ChatGPT-style tutor help or hurt on an exam taken without it?”

Bastani et al. 2025, PNAS (peer reviewed). Download · Source

  • Who was studied: About 1,000 students in Grades 9 to 11 at one high school in Turkey, in about 50 maths classes; honours classes were left out.
  • How: Randomised controlled trial (RCT: classes assigned to groups by lottery), preregistered. Over four 90-minute sessions, classes practised with a plain GPT-4 chat that mimicked ChatGPT, with a guarded GPT-4 tutor that had the teacher’s solution and common mistakes and was told to give hints rather than answers, or with the book and notes. Each session ended with a closed-book exam, graded by independent graders.
  • What it found: The plain chat raised practice scores by 48% but lowered the unaided exam by 0.19 SD (standard deviations, the usual unit for comparing test-score gains; our conversion of −0.054 on a 0 to 1 scale, 95% confidence interval −0.35 to −0.04). The guarded tutor raised practice scores by 1.26 SD and left the exam unchanged (−0.01 SD, −0.11 to 0.08). Students copied answers, and GPT-4 solved only 51% of these problems correctly.
  • Why it matters for us: A coach that hands over answers can lower what a JEE student can do alone, and practice accuracy inside the coach will overstate learning.
  • Where we use it: Meta-analysis section 3.9, as one of the seven pooled tutor trials and as the evidence of harm; section 3.10 and section 4.1 point 5, for practice scores misleading; What works, Summary and sections c) and d); AI tutors literature review; Strand A (richer countries) literature review.

Measuring the Impact of AI Tutoring, from Near Transfer to Standardised Tests. Does a supervised AI tutor move a standardised maths test?

Section titled “Measuring the Impact of AI Tutoring, from Near Transfer to Standardised Tests. Does a supervised AI tutor move a standardised maths test?”

LearnLM Team, Google and Eedi 2026, technical report by the tool’s developers. Download · Source

  • Who was studied: 1,179 students in Years 8 to 10 in 58 classes at 10 UK state secondary schools, over 10 weeks; 66% of the sample were boys.
  • How: Student-level RCT with four arms, preregistered. After a wrong answer on a diagnostic question, students got either Eedi’s standard remediation (videos, fluency exercises, a misconception hint), AI tutoring with the question’s context, AI tutoring with personalised context, or an expert human tutor. Gemini drafted the AI messages and an expert tutor approved every one. The main outcome was Renaissance Star Maths, a standardised test run and scored outside the trial.
  • What it found: On Star Maths the AI tutor arm differed by −0.01 SD (95% credible interval −0.15 to 0.14); the personalised-context arm by +0.02 (−0.12 to 0.16); the expert human tutor by −0.09 (−0.25 to 0.07). Closer to the lesson the tutor did help: students corrected the mistake in front of them 56.9% of the time against 43.4%, got a near-transfer question right 58.4% against 50.5%, and got the item right when it returned 7 or more days later 59.9% against 47.6%. Tutored students covered fewer units (99 against 132).
  • Why it matters for us: This is the trial built most like the coach, and its gains on the coached questions did not reach a whole standardised test; we should expect the same gap between coached chapters and JEE.
  • Where we use it: Meta-analysis section 2 (added after the search date), section 3.9 as one of the seven pooled trials, section 3.10 as the trial built most like the coach, and Appendix B; What works, section c).

AI Tutoring Can Safely and Effectively Support Students. Is a human-supervised AI tutor as good as a human tutor?

Section titled “AI Tutoring Can Safely and Effectively Support Students. Is a human-supervised AI tutor as good as a human tutor?”

LearnLM Team, Google and Eedi 2025, arXiv exploratory trial written by the model’s developer. Download · Source

  • Who was studied: 165 students in Years 9 and 10 maths at 5 UK schools, with 17 expert tutors.
  • How: Exploratory RCT on the Eedi platform. Students who needed help got either a static hint or chat tutoring, and tutoring was randomised between an expert human tutor writing alone and LearnLM drafting messages that the tutor approved or edited. The outcome was solving a new problem on the next topic.
  • What it found: Students tutored by supervised LearnLM solved the new problem 66.2% of the time against 60.7% with a human tutor alone (5.5 percentage points, 95% interval −1.4 to 12.4). Tutors approved 76.4% of the AI’s drafts with no or minimal edits. The model made 5 factual errors in 3,617 messages when given the item, the wrong answer and the misconception.
  • Why it matters for us: An AI that works from the item’s verified key, with a teacher able to step in, did at least as well as a human tutor; the comparison was a human tutor, not usual teaching.
  • Where we use it: What works, Summary, for a teacher stepping in where the software gets it wrong; AI tutors literature review, on human-in-the-loop tutoring and grounding in a verified key; AI authoring literature review, on the error rate of tutor messages.

Teaching with Gemini. Does Gemini Guided Learning in class raise maths scores in Sierra Leone?

Section titled “Teaching with Gemini. Does Gemini Guided Learning in class raise maths scores in Sierra Leone?”

LearnLM Team, Google and Fab AI 2026, technical report run and written by the tool’s developer. Download · Source

  • Who was studied: 1,763 students in Grades 7 and 8, in 48 classrooms at 12 government-supported schools in Port Loko, Sierra Leone.
  • How: Classroom-randomised RCT within school and grade, preregistered. Treated classes used Gemini Guided Learning in half of their maths lessons, about 90 minutes a week, two students to a device, structured by the teacher; control classes had standard teaching for the same time, and both arms got the same teacher training. The test was curriculum-aligned and built and run by Oxford MeasurEd, independent of the developer.
  • What it found: After 8 to 9 weeks, 0.258 SD by intention to treat (counting every student assigned, whether or not they used it; 95% interval 0.03 to 0.49), and 0.380 SD for students who were treated. 69% of students met the 12-hour dose. Gains were larger for students who started stronger. The Gemini model changed during the trial.
  • Why it matters for us: A tutor used inside timetabled lessons with a teacher present gave a gain on an independent test, which is the setting a Centre of Excellence can offer.
  • Where we use it: Meta-analysis section 3.9, as one of the seven pooled trials; section 3.11, on stronger students gaining more; section 4.2, among the closest settings to ours; Strand B (poorer countries) literature review.

One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment. Does adding an AI tutor to Khan Academy raise maths scores?

Section titled “One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment. Does adding an AI tutor to Khan Academy raise maths scores?”

Oreopoulos and Low 2026, EdWorkingPaper 26-1551 (also NBER Working Paper 35620). Download · Source

  • Who was studied: Middle-school students in remedial maths periods at 18 schools in Tennessee, US, over two school years: 6,902 student-terms in 53 clusters.
  • How: Cluster RCT, preregistered, with grades randomised within schools. Treated grades used Khan Academy with Khanmigo, set to coach rather than give answers, in daily remedial blocks; the comparison was the usual remediation, often other adaptive software. The outcome was NWEA MAP, a standardised test.
  • What it found: 1.26 national percentile ranks per term (SE 0.60), or 0.050 SD per term; 0.06 to 0.08 SD a year, and an implied 0.14 SD for a full year of participation. That is about the same as Khan Academy practice without AI. 96% of students tried the tutor, but the median student messaged it in only 17% of practice sessions with a mistake, and most messages were bare answers or clicks on suggested prompts.
  • Why it matters for us: Giving students an AI tutor does not mean they use it; the coach has to come to the student at the mistake rather than wait to be asked.
  • Where we use it: Meta-analysis section 3.9, as one of the seven pooled trials; AI tutors literature review, on access not being use; Adaptive platforms literature review.

Making AI Tutoring Productive. Does an AI tutor help more when students must master each skill before moving on?

Section titled “Making AI Tutoring Productive. Does an AI tutor help more when students must master each skill before moving on?”

Oreopoulos, Liut, Sungu and Low 2026, EdWorkingPaper 26-1552 (also NBER Working Paper 35621). Download · Source

  • Who was studied: More than 6,000 middle-school students in Hamilton County, Tennessee, US, practising maths on the NUMI platform in March 2026.
  • How: Individually randomised two-by-two experiment: a guarded AI tutor that gave no direct answers, or none, crossed with a mastery rule (three correct in a row before moving on) or free progression. Learning was measured on a test item one week later.
  • What it found: With the AI, students attempted fewer questions, worked more slowly and more accurately, and were more often right on the next attempt after a mistake. The mastery rule alone gave no delayed gain. The AI inside the mastery rule gave about 3 to 4.2 percentage points on the practised item a week later, marginally significant, and no difference on an unpractised item.
  • Why it matters for us: AI help is most useful right after a mistake, and a rule that makes the student earn progress gives them a reason to use the help to understand.
  • Where we use it: AI tutors literature review, for the design rule of attempt first, then hints, then the worked solution.

Evaluating AI Tutoring at the Speed of Innovation. Does an AI tutor that marks long answers help GCSE science revision?

Section titled “Evaluating AI Tutoring at the Speed of Innovation. Does an AI tutor that marks long answers help GCSE science revision?”

Harrison et al. 2026, arXiv; an evaluation funded by the vendor. Download · Source

  • Who was studied: GCSE students in Years 10 and 11 at English secondary schools, studying biology, chemistry and physics: 644 analysed of 929.
  • How: Student-randomised trials run across several schools by practitioners. Students had four weekly 30-minute sessions with Medly, an AI tutor that marks extended answers, against self-directed revision. The outcome was a curriculum-aligned post-test written by the evaluator.
  • What it found: Hedges g = 0.33 (a standardised effect size, read like SD; 95% interval 0.18 to 0.48) after four weeks; physics 0.31 (0.11 to 0.51). Attrition was 30.7%.
  • Why it matters for us: AI marking of written working during revision gave a moderate gain for secondary science students, though on a short horizon and a non-independent test.
  • Where we use it: Meta-analysis section 3.9, as one of the seven pooled trials; AI tutors literature review, on effects in secondary settings.

From Chalkboards to Chatbots. Did an after-school GPT-4 English programme raise scores in Nigeria?

Section titled “From Chalkboards to Chatbots. Did an after-school GPT-4 English programme raise scores in Nigeria?”

De Simone et al. 2025, World Bank Policy Research Working Paper 11125. Download · Source

  • Who was studied: Senior Secondary 1 students (about 15 years old) who volunteered, at nine public schools in Benin City, Nigeria: 759 analysed of 1,328 randomised.
  • How: Student-level RCT. Treated students had twelve 90-minute after-school sessions over six weeks, using Microsoft Copilot (GPT-4) in pairs with teacher prompts and supervision; the control group had no programme and no extra time.
  • What it found: 0.31 SD on the researchers’ endline test (95% interval 0.18 to 0.44), 0.238 in English, and 0.206 SD (0.08 to 0.34) on the school’s own third-term English exam. More treated than control students completed (64% against 50%); bounds that allow for this put the total between 0.255 and 0.327. Gains were larger for higher-performing and female students.
  • Why it matters for us: Supervised, shared-device AI sessions worked in a low-resource school system, but the gain on the regular school exam was about two-thirds of the gain on the researchers’ test.
  • Where we use it: Meta-analysis section 3.9, as one of the seven pooled trials (one of the two rated high-risk); section 3.11, on stronger students gaining more; section 4.2, among the closest settings to ours; AI tutors literature review; India context literature review; Strand B (poorer countries) literature review.

Effective and Scalable Math Support. Does a WhatsApp maths tutor raise scores in Ghana?

Section titled “Effective and Scalable Math Support. Does a WhatsApp maths tutor raise scores in Ghana?”

Henkel et al. 2024, arXiv; the authors include the provider, Rising Academies. Download · Source

  • Who was studied: Students in Grades 3 to 9 at 11 Rising Academies schools in Ghana: 477 of 637 kept to the end (25% attrition).
  • How: School-randomised trial (5 treatment and 6 control schools). Treated students used Rori, a WhatsApp maths tutor giving micro-lessons with a hint and then a worked solution, twice a week for 30 minutes in study hall on phones, for eight months. The outcome was a 35-item foundational maths test.
  • What it found: Cohen’s d = 0.36 (a standardised effect size, read like SD). With our correction for having only 11 schools, the 95% interval is 0.03 to 0.69. The content was well below grade level.
  • Why it matters for us: A low-cost tutor on shared phones in a timetabled slot can work, but this was basic maths far below JEE level.
  • Where we use it: Meta-analysis section 3.9, as one of the seven pooled trials (one of the two rated high-risk); AI tutors literature review; India context literature review, on shared devices and low connectivity.

AI Tutoring Outperforms In-Class Active Learning. Can an AI tutor at home beat an active physics class?

Section titled “AI Tutoring Outperforms In-Class Active Learning. Can an AI tutor at home beat an active physics class?”

Kestin et al. 2025, Scientific Reports (peer reviewed). Download · Source

  • Who was studied: 194 undergraduates in an introductory physics course for life-science students at Harvard, US.
  • How: Crossover RCT over two lessons (surface tension, fluids): each student had one lesson from a GPT-4 tutor at home and one in an active-learning class. The tutor had step-by-step solutions in its prompt, and the platform walked students through each part of the problem in order. Time was not matched: median 49 minutes with the tutor, 60 in class.
  • What it found: 0.63 SD on an immediate post-test written by the researchers (95% interval about 0.40 to 0.86, our approximation), and 0.73 to 1.3 SD by quantile regression. The test was near its ceiling, and the comparison was already a strong form of teaching.
  • Why it matters for us: A tutor grounded in a full solution and sequenced step by step works well for first exposure to new material, though for strong university students and on an immediate test only.
  • Where we use it: AI tutors literature review, for using the coach for structured first exposure; Strand A (richer countries) literature review.

The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement. Does a course AI tutor raise university grades?

Section titled “The Effects of Course-Integrated AI Tutoring on Student Performance and Engagement. Does a course AI tutor raise university grades?”

Liu et al. 2026, EdWorkingPaper 26-1598. Download · Source

  • Who was studied: 2,379 undergraduates taught by 30 instructors at a large US public university, Fall 2025.
  • How: RCT with instructors randomised within course blocks. Treated courses got a GPT-4o tutor built on the course materials; most instructors left it in its default direct-answer mode, and only two ended the term in tutor mode.
  • What it found: Final course grades fell by 0.27 to 0.37 SD. Participation in set course activities fell by 0.90 SD, and AI users said they asked instructors fewer questions. Losses were larger for first-generation students.
  • Why it matters for us: An answer tool within easy reach can displace practice and contact with teachers; the coach should send students back to teachers, and its default must not be to answer.
  • Where we use it: Meta-analysis section 3.9, as evidence of harm from answer-giving; AI tutors literature review, on defaults, displacement and who is harmed.

ChatGPT-Generated Help Produces Learning Gains Equivalent to Human Tutor-Authored Help. Can AI-written hints teach as well as human ones?

Section titled “ChatGPT-Generated Help Produces Learning Gains Equivalent to Human Tutor-Authored Help. Can AI-written hints teach as well as human ones?”

Pardos and Bhandari 2024, PLOS ONE (peer reviewed). Download · Source

  • Who was studied: 274 US adults recruited on Mechanical Turk, working algebra and statistics problems in an open-source tutor.
  • How: Online RCT with three hint conditions: worked-solution hints written by ChatGPT 3.5, hints written by human tutors, or no help. The ChatGPT hints for 74 problems were checked for correctness.
  • What it found: Learning gains of 17.0 percentage points with ChatGPT hints, 11.6 with human hints and 1.9 with none; ChatGPT and human hints did not differ significantly. But 32% of the raw ChatGPT hints failed a correctness check (25% to 47% by topic). Taking the majority answer of 10 samples cut errors to 0% to 2% in algebra and 13% in statistics, and a human check took 37.6 seconds per hint.
  • Why it matters for us: AI-written worked solutions can teach once wrong ones are removed, so every generated step needs checking against Avanti’s verified key before a student sees it.
  • Where we use it: AI authoring literature review, for re-solving several times and checking against the key; Step-based tutoring literature review; AI tutors literature review, on grounding every problem in a verified key.

AI Meets the Classroom: When Do Large Language Models Harm Learning? Does free access to an LLM help people learn to code?

Section titled “AI Meets the Classroom: When Do Large Language Models Harm Learning? Does free access to an LLM help people learn to code?”

Lehmann, Cornelius and Sting 2025, arXiv working paper. Download · Source

  • Who was studied: University and graduate learners in the Netherlands learning Python: 108 lab subjects in the first study, and 113 students in two graduate courses.
  • How: Two preregistered, incentivised lab experiments comparing LLM access with none, plus a field study that used ChatGPT outages during two courses.
  • What it found: No overall effect of LLM access in the labs. In exploratory analysis, learners who used the LLM to generate solutions covered more topics with less understanding, while those who asked it for explanations understood more. Access helped learners with high prior knowledge and hurt those with low prior knowledge.
  • Why it matters for us: Left to choose, students let the AI do the work, and the weaker students are the most exposed; the coach has to steer them towards explanations.
  • Where we use it: AI tutors literature review, on answer-giving, who benefits, and asking the student to do the thinking.

Beware of Metacognitive Laziness. Does ChatGPT help students learn while they revise an essay?

Section titled “Beware of Metacognitive Laziness. Does ChatGPT help students learn while they revise an essay?”

Fan et al. 2024, arXiv working paper. Download · Source

  • Who was studied: 117 university students in China, revising an essay.
  • How: Randomised lab study with four kinds of support: ChatGPT, a human expert, a writing checklist, or none.
  • What it found: The ChatGPT group improved its essay score the most, but knowledge gain and transfer did not differ between groups, and neither did intrinsic motivation. Students with ChatGPT took fewer self-regulation steps, such as planning and checking.
  • Why it matters for us: AI can produce a better piece of work without more learning, so the coach should ask the student to plan, predict and check rather than doing it for them.
  • Where we use it: AI tutors literature review, on practice performance against exam performance and the rule to measure learning only without the AI.

Little Impact of ChatGPT on High School Test Scores. Did everyday student ChatGPT use change US test scores?

Section titled “Little Impact of ChatGPT on High School Test Scores. Did everyday student ChatGPT use change US test scores?”

Huntington-Klein 2026, arXiv working paper. Download · Source

  • Who was studied: US high-school students in aggregate: state and metro-area test scores across the US, compared with ChatGPT search activity in 210 metro areas, 2023-2025.
  • How: Observational. Areas where ChatGPT activity drops most in the summer holidays are treated as areas with heavy school use of ChatGPT, and their test-score changes are compared with other areas.
  • What it found: No clear effect. One SD more school-related ChatGPT use changed test scores by −0.0096 SD (95% interval −0.029 to 0.010), not statistically different from zero; the author bounds any harm at about 0.04 SD.
  • Why it matters for us: Unsupervised AI use has not shown a large average harm on test scores, but averages can hide harm in students who let the AI do practice for them.
  • Where we use it: AI tutors literature review, in the evidence table and as weak evidence.

Your Brain on ChatGPT. Brain activity and recall when adults write essays with AI.

Section titled “Your Brain on ChatGPT. Brain activity and recall when adults write essays with AI.”

Kosmyna et al. 2025, arXiv preprint (not peer reviewed), MIT Media Lab. Download · Source

  • Who was studied: 54 adults aged 18-39 from five Boston-area universities; 18 of them returned for a fourth session. Task: essay writing.
  • How: Lab experiment. Participants were assigned to write essays with ChatGPT, with a search engine, or with no tool, over three sessions, with brain activity recorded by EEG (electrodes on the scalp); in session 4 some groups switched tools.
  • What it found: The ChatGPT group showed the weakest brain connectivity and felt less ownership of their essays. In session 1, 83% (15 of 18) of the ChatGPT group could not correctly quote their own essay minutes after writing it, against 11% in the other two groups.
  • Why it matters for us: If the coach writes or solves for the student, the student may not remember or own the work; the coach should make the student do the thinking.
  • Where we use it: AI tutors literature review, in the evidence table as a small, unreviewed signal on effort and recall.

Tutoring software and computer-assisted learning

Section titled “Tutoring software and computer-assisted learning”

Effectiveness of Intelligent Tutoring Systems. How much does tutoring software help, and on whose test?

Section titled “Effectiveness of Intelligent Tutoring Systems. How much does tutoring software help, and on whose test?”

Kulik and Fletcher 2012, Institute for Defense Analyses Document D-4664 (precursor of their 2016 Review of Educational Research article). Download · Source

  • Who was studied: 45 evaluations of intelligent tutoring systems (software that guides a student step by step) in US schools, colleges and the military.
  • How: Meta-analysis (a statistical pooling of many studies’ results) comparing tutoring software with conventional teaching, broken down by study features.
  • What it found: A mean gain of 0.60 SD (median 0.63). But 0.72 SD on locally written tests (33 evaluations) against 0.08 SD on standardised tests (7); for Cognitive Tutor, 0.72 against 0.10. Studies with 80 or fewer participants averaged 0.76 and those with over 250 averaged 0.30; studies of 17 weeks or more averaged 0.29. K-12 studies averaged 0.43.
  • Why it matters for us: Gains on a system’s own quizzes overstate transfer, so the coach must be judged on independent mocks and JEE, not on its own tests.
  • Where we use it: Meta-analysis Planning ranges, as an anchor for step-based tutoring software at scale; sections 3.10 and 3.11, on who wrote the test and on study size; Adaptive platforms literature review; Step-based tutoring literature review.

Online Mathematics Homework Increases Student Achievement. Does feedback-giving homework software raise Grade 7 maths?

Section titled “Online Mathematics Homework Increases Student Achievement. Does feedback-giving homework software raise Grade 7 maths?”

Roschelle, Feng, Murphy and Mason 2016, AERA Open (peer reviewed). Download · Source

  • Who was studied: 2,850 Grade 7 students (2,728 tested) at 43 schools in Maine, US.
  • How: School-randomised RCT. Treated schools used ASSISTments maths homework, which gives immediate feedback to students and reports to teachers; control schools did their usual homework. The outcome was TerraNova, a standardised test not aligned to the homework.
  • What it found: Hedges g = 0.18 at the end of the second year of use. The gain was 0.29 for students at or below the median and 0.12 for those above. Students used it for about 14 hours a year against the 18 to 24 planned.
  • Why it matters for us: Plain practice with instant feedback and teacher reports moved an independent test at scale, and helped weaker students most.
  • Where we use it: Meta-analysis section 3.11, on software at scale and on results splitting by prior level; Adaptive platforms literature review.

Effectiveness of Cognitive Tutor Algebra I at Scale. Does a tutoring-software curriculum raise algebra scores across many schools?

Section titled “Effectiveness of Cognitive Tutor Algebra I at Scale. Does a tutoring-software curriculum raise algebra scores across many schools?”

Pane, Griffin, McCaffrey and Karam 2014, Educational Evaluation and Policy Analysis (peer reviewed); read through the RAND working-paper version. Download · Source

  • Who was studied: High-school Algebra I students at 147 schools in seven US states; 11,066 in the final high-school sample.
  • How: Cluster RCT comparing the Cognitive Tutor Algebra I curriculum, which combines software and classroom lessons, with usual teaching. The outcome was a standardised algebra proficiency exam.
  • What it found: No significant effect in the first year (−0.07 to −0.17 SD, not significant) and 0.12 to 0.21 SD in the second year, as schools learned to run it. Attrition was 28% to 36%. Our figures come from the RAND addendum; the journal article was not opened.
  • Why it matters for us: Even a well-built tutoring system needed a year of implementation before it paid off, so the coach’s first year should be judged with that in mind.
  • Where we use it: Meta-analysis section 3.11, on software at scale; Adaptive platforms literature review.

Remedying Education: Evidence from Two Randomized Experiments in India. How long does a gain from computer-assisted maths last?

Section titled “Remedying Education: Evidence from Two Randomized Experiments in India. How long does a gain from computer-assisted maths last?”

Banerjee, Cole, Duflo and Linden 2007, Quarterly Journal of Economics (peer reviewed); archived as NBER Working Paper 11904. Download · Source

  • Who was studied: Grade 4 students in municipal schools in Vadodara, India; about 4,700 in the follow-up.
  • How: School-randomised RCT. We use the computer-assisted learning arm: two hours a week of maths games on shared computers, against none. The outcome was a maths test set by Pratham and the researchers.
  • What it found: 0.35 SD after one year and 0.47 SD after two. One year after the programme ended the gain was 0.097 SD (SE 0.053), about a fifth of its peak. Gains were equal across ability levels.
  • Why it matters for us: Gains from practice software can fade fast once it stops, so the coach should keep spaced review running through to JEE.
  • Where we use it: Meta-analysis section 3.11, on fade-out.

Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India. Does adaptive software raise maths in Delhi after-school centres?

Section titled “Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India. Does adaptive software raise maths in Delhi after-school centres?”

Muralidharan, Singh and Ganimian 2019, American Economic Review (peer reviewed); archived as NBER Working Paper 22923. Download · Source

  • Who was studied: 619 students in Grades 6 to 9 who applied for after-school Mindspark centres in Delhi, India.
  • How: RCT by individual lottery. Winners got 45 minutes of adaptive Mindspark software and 45 minutes of group instruction, six days a week, for 4.5 months. Outcomes were a researcher test scaled by item response theory and the students’ school exams.
  • What it found: Maths rose by 0.36 SD by intention to treat (section 3.11 of our meta-analysis gives 0.37) and Hindi by 0.22 SD; 90 days of attendance implied 0.59 SD. The maths gains were only on items below grade level, with no significant gain on grade-level school maths exams. Attendance was 58%.
  • Why it matters for us: Software that meets students at their level fills gaps but may not move a grade-level exam; JEE is a grade-level-and-above exam.
  • Where we use it: Meta-analysis section 3.11 and Appendix B, on adaptive software not moving the grade-level exam; India context literature review; Adaptive platforms literature review; Techniques that work literature review.

Adapting for Scale: Experimental Evidence on Technology-Aided Instruction in India. Does Mindspark work inside government schools?

Section titled “Adapting for Scale: Experimental Evidence on Technology-Aided Instruction in India. Does Mindspark work inside government schools?”

Muralidharan and Singh 2025, NBER Working Paper 34205. Download · Source

  • Who was studied: Students in Grades 1 to 8 at 80 government schools in Rajasthan, India (40 treated): 8,733 students in our meta-analysis table, about 6,500 of them in treated schools.
  • How: School-randomised RCT of Mindspark in school-day computer labs, with devices shared in pairs, over 18 months. Outcomes were an independent test and school exams.
  • What it found: 0.22 SD in maths and 0.20 SD in Hindi, with no significant effect on school exams. Gains tracked time on the platform, and usage roughly halved when one lab coordinator was spread across 3 to 4 schools.
  • Why it matters for us: Shared devices can work, but the adult who runs the slot carries the usage; the coach needs a named teacher and protected time at each Centre.
  • Where we use it: Meta-analysis section 3.11, on software in poorer countries; India context literature review, on shared devices and the adult in the room.

How In-School Supervised Ed-Tech Support Produces Massive Learning Gains. Does an adult who protects practice time make Khan Academy work in India?

Section titled “How In-School Supervised Ed-Tech Support Produces Massive Learning Gains. Does an adult who protects practice time make Khan Academy work in India?”

Oreopoulos, Keyes-Krysakowski and Agarwal 2026, NBER Working Paper 34683; co-authored with Khan Academy’s research partners, not yet peer reviewed. Download · Source

  • Who was studied: Students in Grades 6 to 8 at residential government middle schools in Uttar Pradesh, India: 74 of 83 schools analysed (24 treatment, 50 control), 7,888 students at endline.
  • How: School-randomised trial. Every school had free Khan Academy; treated schools also got a lab-in-charge who protected and supervised practice sessions. The outcome was an independently administered maths test after 31 weeks.
  • What it found: Practice rose from 7.2 to 47.4 minutes a week. Maths scores rose by 0.33 SD (SE 0.111) on endline data alone and 0.44 SD (SE 0.100) with baseline scores and controls, by intention to treat; 0.40 to 0.47 SD for students treated. 41% to 46% of students were missing at endline.
  • Why it matters for us: This is the closest setting to our Centres, residential government schools in India, and it says the adult running the slot matters more than the software.
  • Where we use it: What works, Summary, for a teacher in the room keeping students practising; Meta-analysis section 3.11 and section 4.2; Adaptive platforms literature review; AI tutors literature review, on access not being use; India context literature review.

The Effectiveness of Technology-Supported Personalised Learning in Low- and Middle-Income Countries. How much does personalised software help in poorer countries?

Section titled “The Effectiveness of Technology-Supported Personalised Learning in Low- and Middle-Income Countries. How much does personalised software help in poorer countries?”

Major, Francis and Tsapali 2021, British Journal of Educational Technology (peer reviewed). Download · Source

  • Who was studied: 53,029 school-age students in 16 RCTs from 2007 to 2020, in five low- and middle-income countries, many of them in China.
  • How: Meta-analysis of RCTs of technology-supported personalised learning against a control, mostly on standardised or independent tests.
  • What it found: g = 0.18 (95% interval 0.12 to 0.24), with high variation between studies (I² 84%). Software that adapts to the student’s level gave 0.35. Trials by independent research groups gave 0.26 against 0.14 for one large Chinese group.
  • Why it matters for us: It sets a realistic range for practice software in poorer settings, well below the large gains in some single trials.
  • Where we use it: Meta-analysis Planning ranges, as the anchor for adaptive practice software in poorer countries; section 3.11.

Tutoring at scale and reading the evidence

Section titled “Tutoring at scale and reading the evidence”

The Impressive Effects of Tutoring on PreK-12 Learning. How much does human tutoring help, on average?

Section titled “The Impressive Effects of Tutoring on PreK-12 Learning. How much does human tutoring help, on average?”

Nickow, Oreopoulos and Quan 2020, NBER Working Paper 27476. Download · Source

  • Who was studied: PreK-12 students in randomised trials of tutoring, mostly in the US and UK: 96 studies with 732 effect sizes.
  • How: Meta-analysis of tutoring RCTs in reading and maths.
  • What it found: Tutoring raised scores by 0.37 SD on average. Studies with over 400 students gave 0.25, against 0.45 for those under 50. Teachers as tutors gave 0.50. Tutoring during the school day did about twice as well as after school. Only 7 or 8 studies covered Grades 6 to 11.
  • Why it matters for us: Human tutoring is the strongest evidence in our review, and in-school tutoring beats after-school tutoring, which supports running the coach in timetabled Centre time.
  • Where we use it: What works, Summary, on tutoring during the school day; Meta-analysis section 3.11 and Appendix B, for human tutoring across all trials; Teacher, student and AI literature review; Techniques that work literature review, on the two-sigma claim; Adaptive platforms literature review.

What Impacts Should We Expect from Tutoring at Scale? How tutoring gains shrink as programmes grow.

Section titled “What Impacts Should We Expect from Tutoring at Scale? How tutoring gains shrink as programmes grow.”

Kraft, Schueler and Falken 2024, EdWorkingPaper 24-1031 (Brown University). Download · Source

  • Who was studied: 265 randomised trials of tutoring programmes for US school students (282 in the October 2024 cover abstract), maths and reading, all grades.
  • How: Meta-analysis, with the results broken down by how many students each programme served.
  • What it found: Across all trials tutoring raised scores by 0.42 SD. But the gain falls with size: 0.55 SD for programmes under 100 students, 0.21 SD for 400-999 students and 0.16 SD for 1,000 or more, on standardised tests. Tutoring outside the school day is associated with effects about 0.19 SD smaller at scale.
  • Why it matters for us: Even strong human tutoring delivers about 0.16-0.21 SD at scale, so an AI coach used a few hours a week should be planned against a smaller target.
  • Where we use it: What works, Summary, for what human tutoring achieves at scale and in school hours; Meta-analysis Planning ranges, as the anchor for human tutoring at scale; section 3.11 and Appendix B; Teacher, student and AI literature review.

Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. Does AI advice to human tutors help their students?

Section titled “Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise. Does AI advice to human tutors help their students?”

Wang, Ribeiro, Robinson, Loeb and Demszky 2024 (version 3, 2026), arXiv working paper. Download · Source

  • Who was studied: 783 tutors and 1,013 students of a virtual tutoring provider, mostly in Grades 3 to 6, from low-income families in the southern US.
  • How: Preregistered RCT, with tutors randomised. Treated tutors saw LLM-suggested moves during live chat sessions and chose whether to use them, for about seven weeks.
  • What it found: Students of treated tutors were 4 percentage points more likely to pass the session’s exit ticket (62% to 66%), and 9 points more for students of lower-rated tutors. No significant end-of-year test effect was detected; the authors’ rough estimate for a full year is 0.024 SD. The tool cost about USD 20 per tutor per year.
  • Why it matters for us: AI that advises a teacher, who keeps the choice, raises quality cheaply, most for less experienced teachers; this fits the coach’s briefs to Centre teachers.
  • Where we use it: Teacher, student and AI literature review, for writing teacher briefs as options and reading proximal gains as early signals only; AI tutors literature review, on human-in-the-loop tutoring.

Effect of Artificial Intelligence on Learning: A Meta-Meta-Analysis. What is left of AI’s effect on learning after correcting for publication bias?

Section titled “Effect of Artificial Intelligence on Learning: A Meta-Meta-Analysis. What is left of AI’s effect on learning after correcting for publication bias?”

Bartoš, Bujak, Martinková and Wagenmakers 2026, PsyArXiv preprint, not peer reviewed. Download · Source

  • Who was studied: Learners at all levels, through 1,840 effect sizes taken from 67 meta-analyses of AI and LLMs in education.
  • How: A re-analysis of the studies inside those meta-analyses, with a Bayesian method that adjusts for publication bias (positive results being more likely to be published).
  • What it found: The unadjusted average of 0.629 (0.572 to 0.685) falls to 0.196 (0.000 to 0.323) after adjustment. Adjusted, academic achievement is 0.10, STEM 0.05, and AI used as a tutor −0.04 (−0.61 to 0.28). Negative results were strongly suppressed, by a factor of about three.
  • Why it matters for us: Headline AI-in-education averages are inflated, so the coach should be planned against small effects.
  • Where we use it: Meta-analysis section 3.9, as an outside check on our pool; section 3.11, on publication bias.

Can AI Improve Household Welfare in Poorer Countries? What do AI tutoring trials add up to, measured without the tool?

Section titled “Can AI Improve Household Welfare in Poorer Countries? What do AI tutoring trials add up to, measured without the tool?”

Dang and Do 2026, IZA Discussion Paper 18984. Download · Source

  • Who was studied: Students in eight AI tutoring comparisons from Ghana, Nigeria, Sierra Leone, Turkey, the US, Germany and Italy, in maths, English and physics.
  • How: A structured critical review with a random-effects pool of AI tutoring with human support or guardrails against usual teaching, counting only tests taken without the tool.
  • What it found: 0.21 SD (95% interval 0.05 to 0.36), with high variation (I² 76%). Without the two high-risk trials from Ghana and Nigeria, 0.16 (−0.04 to 0.37). None of the trials measured retention.
  • Why it matters for us: An independent pool lands close to ours, about 0.16 to 0.21 SD, which is the right scale to plan against.
  • Where we use it: Meta-analysis section 3.9, as an outside check on our pool.

Mechanisms of Effect Size Differences Between Researcher Developed and Independently Developed Outcomes. How much bigger do effects look on the researchers’ own tests?

Section titled “Mechanisms of Effect Size Differences Between Researcher Developed and Independently Developed Outcomes. How much bigger do effects look on the researchers’ own tests?”

Gilbert and Soland 2026, EdWorkingPaper 24-1082. Download · Source

  • Who was studied: 30 randomised trials, mostly in the US, that measured the same intervention on both a researcher-written and an independent test and released item-level data: 45 effect sizes.
  • How: Item-level meta-analysis comparing the two kinds of test within the same studies.
  • What it found: Effects on researcher-written tests were 0.24 SD higher (95% interval 0.17 to 0.31), and 0.15 higher after allowing for item features. The paper reports Wolf and Harbatkin’s finding of a 0.24 to 0.32 SD gap.
  • Why it matters for us: A gain on the coach’s own questions should be roughly halved before it is compared with JEE, and expected smaller still.
  • Where we use it: Meta-analysis sections 3.10 and 3.11, for the rule of thumb to halve a researcher-test effect.