Skip to content
Avanti Docs

Techniques that raise test performance after a test: a meta-analysis

Abstract

Section titled “Abstract”

Background. Students in Avanti’s Centres of Excellence spend 3 to 4 hours a day studying on their own after class. This review asks which techniques, used after a test on the questions a student got wrong, raise scores on later JEE-style questions, and which of them an AI coach could deliver.

Methods. Four parallel reviews, searched on 1 November 2026, covered practice testing, spacing, mixed practice, feedback, worked and faded examples, expertise reversal, self-explanation, erroneous examples, problem-first instruction, contrasting cases, misconception refutation, the attempt rule under negative marking, and tutoring by people, software and AI. We extracted 181 effect sizes from about 115 meta-analyses and controlled studies, each checked in an archived copy of the source. For each technique we report one anchor meta-analysis, with the classroom trials outside it beside it. Only the AI-tutor trials were pooled, by random effects. Each technique gets a planning range on a delayed test of the coached material and, where the evidence allows, on a full exam, rated for certainty in the spirit of GRADE.

Results. Three findings have high certainty: teaching the ideas before practice (discovery without an explanation does worse, d = −0.38), practice testing for formulas and facts (g = 0.53 in classrooms), and human tutoring at scale (0.16 to 0.21 SD on standardised tests). Worked examples help students weak in a chapter (g = 0.48) and hurt strong ones (d = −0.43). Feedback that gives the correct answer is worth 0.32 SD, against 0.05 for right or wrong alone. Practice testing on multi-step problems, which is most of JEE, shows no reliable gain (g = −0.12 on procedures, 0.02 on transfer). Seven school trials of structured AI tutors pool to 0.16 SD (95% CI 0.05 to 0.28; prediction interval −0.21 to 0.54), and to 0.11 on independent tests.

Conclusions. On the coached chapters, a loop of the correct answer and the working, the student’s own error, worked examples for students weak in the chapter and an unaided retry plausibly moves scores by 0.3 to 0.6 SD. On JEE Main, where those chapters are a small share of the paper, we expect about 0.05 to 0.15 SD, with a real chance of zero. No study here was run with Indian students preparing for JEE or NEET.

1. Introduction

Section titled “1. Introduction”

Avanti’s Centres of Excellence are residential, two-year programmes in which Avanti’s teachers deliver about 6 hours of teaching in Physics, Chemistry and Maths each day. Students there score about 16 to 21 JEE Main percentile points higher than students with the same Class-10 marks who received after-school instruction (Avanti Fellows 2026). Outside class they study 3 to 4 hours a day on their own, with Avanti’s modules, NCERT books and reference books.

Avanti’s earlier work putting digital content into the classroom had small or negative effects. Classroom videos with workbooks in Haryana lowered maths scores by about 0.15 SD (de Barros 2022), and the blended programme in Chennai did not move Class X board scores (Chiplunkar, Dhar and Nagesh 2020). This review therefore looks at the time after a test, on the questions a student got wrong, and asks what a coach, a teacher or an AI, should do with it. A JEE-style test is a timed paper of multiple-choice and numerical questions, marked +4 for a right answer and −1 for a wrong one.

The companion note, What works, summarises the findings for the programme. This paper sets out the evidence in full: the methods, the estimate behind each technique, the studies, and the limits. The strand-by-strand literature reviews and the reading library of papers read but not cited hold the wider reading behind it.

1.1 Question and scope

Section titled “1.1 Question and scope”

Question. Which techniques, delivered after a test on the questions a student got wrong, raise performance on later JEE-style questions, and by how much?

Techniques covered. Practice testing, spacing, mixed (interleaved) practice, feedback, worked and faded examples, expertise reversal, self-explanation, erroneous examples, problem-first instruction, contrasting cases, misconception refutation, teaching the attempt rule under negative marking, and tutoring by people, software and AI.

Population of interest. Students in Grades 11 and 12 preparing for competitive STEM entrance exams, many from low-income families, in India.

Outcomes. Two, kept apart throughout:

  • A delayed test of what was coached. New questions on the same idea, days to weeks later, without help. This is what the coach can move directly.
  • A full exam. A full-length mock or JEE Main itself. Every technique shrinks here, because the coached ideas are a small part of the paper and the exam is written by someone else.

2. Methods

Section titled “2. Methods”

Search. Four reviews ran in parallel, one per group of techniques, each with the same instructions:

  1. Check every figure against its primary source.
  2. Find the best meta-analyses and the strongest classroom trials, prioritising secondary-school STEM, real classrooms over the lab, high-stakes exams, and low- and middle-income countries.
  3. Include work from 2020 to 2026.

The searches ran on 1 November 2026. They missed one report released three days earlier, the second UK trial of Gemini tutoring on the Eedi platform (LearnLM Team, Google and Eedi 2026): the reviews searched by topic, and none went back to the research groups whose earlier trials they already held. It was added on 2 November and is in the pools below.

Inclusion. Meta-analyses and randomised or quasi-experimental studies with a comparison group. Narrative reviews were used for context only. A number is cited only if it was read in an archived copy of the source. Sources read as abstracts only are marked; their pooled figures are not treated as verified. Sources behind publisher bot checks were not opened and are not used.

Extraction. For each estimate the review records (Appendix A):

  • design, population and country
  • comparison, outcome and delay
  • effect size and its 95% CI or standard error
  • number of studies and participants
  • heterogeneity, any bias-adjusted estimate and moderators
  • whether it is independent of the other rows, meaning not already inside a pooled meta-analysis in the table

Effect metric. Standardised mean differences (Hedges’ g or Cohen’s d), in SD units. Where a paper reported a coefficient, it was divided by the control-group SD; the conversion is noted in the table.

Synthesis. Most techniques already have a good meta-analysis, and most classroom trials sit inside one. Pooling a meta-analysis with the studies it already contains would count them twice, so three rules apply:

  • For each technique, one anchor. The most relevant meta-analysis, preferring classroom samples and secondary school, is reported as published, with any bias-adjusted estimate beside it.
  • Classroom trials outside the anchor are reported next to it, not pooled into it. When they disagree with it, they move the planning range.
  • Formal pooling only where independent trials answer one question. Only the AI-tutor trials met that bar. They are pooled by random effects (DerSimonian-Laird), with a Hartung-Knapp-Sidik-Jonkman interval and a 95% prediction interval (Appendix A).

Planning range. For each technique the review gives a range for a delayed test of what was coached and, where it can, one for a full exam such as JEE Main. These are judgements drawn from the evidence, not confidence intervals. Each one says what it rests on.

Certainty. Each range is rated High, Moderate, Low or Very low, in the spirit of GRADE. A rating starts at High for a classroom meta-analysis and drops a level for each of:

  • mostly lab or university samples
  • signs of publication bias, or no bias check at all
  • high heterogeneity, or a prediction interval that crosses zero
  • outcomes written by the researchers rather than independent tests
  • a single small study

3. Results

Section titled “3. Results”

The techniques in plain terms, by strength of evidence

Section titled “The techniques in plain terms, by strength of evidence”

1. A strong case to try today. Consistent evidence, little cost, nothing to build.

Technique What it means On a JEE topic Why it is here
A clear lesson first The ideas are taught directly before any practice A lesson on Gauss’s law before the problem set Leaving students to discover the ideas does worse, with high certainty
Worked examples, for weak students A fully solved problem the student studies step by step before solving one A solved projectile problem, every step shown and labelled 55 studies in maths, and the same learning in about half the time
Straight to problems, for strong students Skip the worked examples once a student knows the chapter A student scoring well on kinematics goes straight to the problem set Worked examples make strong students learn less
Feedback that gives the correct answer After a wrong answer, show the right one and the working, not just a cross “Wrong, it is (C)”, with the solution Most of the value of feedback; a cross alone does almost nothing
Practice testing, for formulas and facts Answering questions from memory instead of rereading notes A closed-book quiz on the rotational motion formulas The best-evidenced technique there is, for things that can be recalled
Spacing Spreading practice on a topic over days and weeks instead of one block Three short sets on electrostatics across three weeks, not one long evening Reliable in real classrooms, if smaller than in the lab
Human tutoring at scale Regular one-to-one or small-group tutoring by trained tutors inside a school programme Three 30-minute sessions a week in groups of two or three High certainty on standardised tests; the cost is people, not evidence

2. Experiments worth trying. Promising, but either the evidence is thin, it comes from a different setting, or it has not shown up on real exams. Try with a comparison group.

Technique What it means On a JEE topic Why it is here
Retry a new question, unaided After seeing the solution, try a fresh question of the same kind with no help A second projectile question, set the next day Carries about half the classic tutoring effect, but never tested on its own
Explanation on top of the correct answer Say why the right answer is right and why the chosen one is tempting “Wrong, it is (C), because the field inside a conductor is zero” Adds a little over the correct answer; in maths transfer questions the plain answer did better
Refuting the misconception Naming the wrong idea, showing where it fails, then giving the right one “Many think a heavier object falls faster. Here is why it does not.” Works in reading studies with university students; only 4 studies in secondary school
Faded worked examples The same solved problem with steps removed one at a time for the student to fill in The projectile problem with the last step blank, then the last two Small trials only, all pointing the same way
Self-explanation prompts A short question at a key step asking the student why it holds “Which law gives this equation?” with three options to choose from Helps, mostly for concepts, and takes time away from practice
Mixed (interleaved) practice A set that mixes question types, so the student has to choose the method Kinematics, work-energy and momentum questions shuffled together Helps on topic tests, not yet on real exams
Problem first, for concepts The student attempts a new kind of problem before being taught it Try to predict what a second lens does, then the lesson Helps understanding of ideas, not procedures
Structured AI tutor An AI that follows a set lesson and holds back the answer, unlike a general chatbot Walks a student through one wrong answer, step by step Six school trials average about 0.2 SD, with a wide spread
Step-based tutoring software Software that checks each line of the working, not only the final answer Flags the line where a sign was dropped Real but small gains on standardised tests at scale
Adaptive practice software Software that picks the next question from the student’s answers so far Two misses on limits, so the next question is an easier limit Helps below-grade skills in India; no gain on grade-level exams

3. Weak evidence. Little or no support for JEE-type problems.

Technique What it means On a JEE topic Why it is here
Practice testing, for multi-step problems Quizzing yourself on a procedure you cannot yet do A closed-book attempt at a full mechanics problem before seeing one solved Nothing measurable in real classrooms
Problem first, for procedures Attempting a new kind of calculation before being taught it Try a titration calculation, then the lesson No gain on procedures
Erroneous examples A solution with a planted mistake for the student to find and fix A solved titration with the mole ratio inverted Small effect, and only when the error is then explained
Contrasting cases Two problems side by side that differ only in the feature that matters The same block on a smooth and on a rough incline Promising, but 4 studies with teenagers

Two practices have strong evidence against them: leaving students to discover ideas with no explanation afterwards, and giving worked examples to students who already know the chapter.

Reading the numbers. g and d are effect sizes in standard deviations (SD). An effect of 0.1 SD moves a student at the middle of the class from the 50th to about the 54th percentile; 0.2 SD to about the 58th; 0.4 SD to about the 66th.

Planning ranges

Section titled “Planning ranges”

Planning ranges by technique, on a delayed test of what was coached and on a full exam

Thick bar and dot: planning range and central estimate on a delayed test of what was coached. Thin bar: on a full exam such as JEE Main. Ranges are judgements from the evidence, not confidence intervals.

Two kinds of test. Each technique gets two ranges because it is measured on two different tests.

  • Coached material. A test, days or weeks later, on the ideas the technique was used on: new vector-algebra questions after a vector-algebra lesson. Almost every study measures this.
  • Full exam. A whole paper the coach did not write, such as JEE Main or a full-length mock. The coached ideas are a small share of its marks, so the same gain is diluted across the rest of the paper. Few studies measure this, so these ranges are our extrapolation, and the column is blank where the evidence gives no basis for one.

The gap between the two columns is the main reason to expect small effects on JEE.

Technique Anchor estimate Planning range, coached material Planning range, full exam Certainty
Practice testing: formulas, facts, one-idea checks g = 0.53 classroom (Ruitenburg et al. 2026); 0.50, bias-adjusted 0.43 to 0.48 (Yang et al. 2021) 0.30 to 0.60 0.00 to 0.05 High
Practice testing: multi-step problems g = −0.12 procedural, 0.02 transfer (Ruitenburg et al. 2026) −0.15 to 0.35 0.00 to 0.05 Low
Spacing g = 0.20 [0.03, 0.36] (Ruitenburg et al. 2026) 0.00 to 0.35 0.00 to 0.10 Moderate
Mixed (interleaved) practice g = 0.40 classroom (Ruitenburg et al. 2026); maths 0.34 (Brunmair and Richter 2019) 0.20 to 0.50 0.00 to 0.15 Moderate
Feedback: correct answer over right/wrong 0.32 vs 0.05 (Van der Kleij et al. 2015) 0.10 to 0.40 Moderate
Feedback: explanation over correct answer 0.71 vs 0.49 near items (Mertens, Finn and Lindner 2022) −0.10 to 0.40 Moderate
Worked examples, weak students g = 0.48 [0.36, 0.60], 0.44 trim-and-fill (Barbieri et al. 2023) 0.10 to 0.30 0.00 to 0.10 Moderate
Worked examples, strong students d = −0.43 [−0.65, −0.21] (Tetzlaff et al. 2025) −0.43 to −0.21 Moderate
Faded worked examples no pooled estimate; g = 0.64 and 0.29 (Renkl et al. 2000) 0.20 to 0.30 Low
Self-explanation prompts g = 0.55, high school 0.43 (Bisra et al. 2018) 0.00 to 0.35 Low
Erroneous examples, error explained g = 0.14 (Alemdag et al. 2025, abstract) 0.10 to 0.33 Low
Problem first, then instruction: concepts g = 0.36 [0.20, 0.51] (Sinha and Kapur 2021) 0.20 to 0.51 Moderate
Problem first, then instruction: procedures g = −0.03 [−0.20, 0.15] (Sinha and Kapur 2021) −0.20 to 0.15 Moderate
Contrasting cases d = 0.34 [−0.01, 0.68] adolescents (Alfieri, Nokes and Schunn 2011) −0.01 to 0.68 Low
Naming and refuting the misconception g = 0.41 [0.30, 0.51], 0.28 trim-and-fill (Schroeder and Kucera 2022) 0.15 to 0.30 0.00 to 0.05 Moderate
Discovery without the explanation d = −0.38 [−0.50, −0.25] (Alfieri et al. 2011) −0.50 to −0.25 High
Structured AI tutor 0.16 [0.05, 0.28], seven trials (this review) 0.05 to 0.28 0.00 to 0.20 Low
Step-based tutoring software at scale 0.08 to 0.20 on standardised tests (Kulik and Fletcher 2012; Leite et al. 2025) 0.10 to 0.20 0.10 to 0.20 Moderate
Adaptive practice software, poorer countries 0.18, adaptive 0.35 (Major, Francis and Tsapali 2021) 0.20 to 0.35 −0.05 to 0.05 Moderate
Human tutoring at scale 0.16 to 0.21 on standardised tests (Kraft, Schueler and Falken 2024) 0.15 to 0.21 0.15 to 0.21 High

3.1 Learn the basics in a lesson first

Section titled “3.1 Learn the basics in a lesson first”

Evidence: high. Discovery without the explanation does worse across a large meta-analysis.

Struggle first helps with ideas, not procedures. Solving a problem before instruction beats instruction first on concepts and transfer (g = 0.36 [0.20, 0.51]) but not on procedures (−0.03 [−0.20, 0.15]) (Sinha and Kapur 2021). Their “bias-corrected 0.87” is a p-curve estimate that moves upward and should not be quoted. Discovery without the explanation loses to explicit teaching (d = −0.38, adolescents −0.53), and guided discovery in science gives only 0.11 (Alfieri et al. 2011).

The studies.

Study What it was Who and where Students
Alfieri et al. 2011 Meta-analysis: discovery learning against explicit teaching, and discovery with guidance Children, adolescents and adults; maths, science and other subjects 5,226 learners in 108 studies (unguided); 4,243 in 56 studies (guided)
Sinha and Kapur 2021 Meta-analysis: solving a problem before being taught (productive failure) against teaching first Grade 2 to adults, most comparisons in Grades 6 to 10; mostly maths and science; Asia, North America, Europe 53 studies, 166 comparisons (students not reported)

3.2 Weak in the chapter: solved examples before practice

Section titled “3.2 Weak in the chapter: solved examples before practice”

Evidence: moderate. Strong in maths meta-analyses, smaller in real classrooms, where the dependable benefit is time saved.

Worked examples are the best-evidenced technique for weak students, mostly through time saved. Against problem solving in maths, g = 0.48 [0.36, 0.60] across 55 studies, 0.44 after trim-and-fill (Barbieri et al. 2023). Heterogeneity is extreme (I² = 94%), and a new study could land anywhere from about −1 to +2. Classroom trials are smaller:

  • Booth et al. 2015, 425 Algebra I students randomised within classrooms: assignments that swapped some practice problems for correct and incorrect worked examples with explanation prompts raised end-of-unit scores by 0.10 SD (intent to treat), a gain too small to rule out zero (95% CI −0.06 to 0.26). Students who started with less prior knowledge gained more.
  • McLaren et al. 2016, two Grade 10 to 11 chemistry trials: worked examples, erroneous examples, tutored problems and plain problem solving all produced the same learning, and worked examples took 46 to 69% less time.

For a student short of hours before JEE, the time saving is the dependable benefit.

Practice testing works for what can be recalled, not for long procedures. The classic result holds: a week after study, tested students recalled 56% of a passage against 42% for rereaders, though rereaders were ahead at five minutes (Roediger and Karpicke 2006). The meta-analyses agree on direction: g = 0.50 in a lab pool (Rowland 2014), 0.50 across 574 classroom effects with a bias-adjusted 0.43 to 0.48 (Yang et al. 2021), and against restudy alone 0.33.

The closest match to our students is a 2026 meta-analysis of real primary and secondary classrooms: 53 studies, 7,303 students (Ruitenburg et al. 2026). Retrieval practice gave g = 0.37 overall and 0.52 in secondary school, but −0.12 [−0.61, 0.38] on procedural knowledge and 0.02 on transfer. Testing in maths alone gave g = 0.18, with a CI crossing zero (Murray, Horner and Goebel 2025). Eight experiments on learning multi-step problems from worked examples found no testing benefit (summarised in van Gog and Sweller 2015); a reply argues for a small positive effect (Karpicke and Aue 2015, abstract only). For a procedure the student cannot yet do, studying a solution beats trying to recall one.

The studies.

Study What it was Who and where Students
Barbieri et al. 2023 Meta-analysis of worked examples against problem solving in maths (53 randomised trials, 2 quasi-experiments) Primary to college, mostly middle school and up; 23 US and 20 non-US articles 55 studies, 181 effects (students not reported)
Booth et al. 2015 Classroom trials of the Algebra by Example worked-example assignments, one unit Non-honours Algebra I, high school, five US districts 425 (main trial); 51 in a three-class trial
McLaren et al. 2016 Two classroom trials of six 40-minute sessions: worked examples, problem solving, erroneous examples and tutored problems Grades 10 to 11 chemistry (stoichiometry), US About 155 and 116
Roediger and Karpicke 2006 Lab experiment: testing against restudying a passage, recall after 5 minutes, 2 days and a week US undergraduates 120
Rowland 2014 Meta-analysis of the testing effect Mostly lab studies with undergraduates; word lists and prose; mostly US 61 studies, 159 effects
Yang et al. 2021 Meta-analysis of quizzing in real classrooms Primary school to university 48,582 in 223 studies
Ruitenburg et al. 2026 Meta-analysis of retrieval practice and spacing in real school classrooms Primary and secondary; US, Netherlands, Germany, Brazil and others 3,391 in 31 studies (retrieval)
Murray, Horner and Goebel 2025 Meta-analysis of testing and spacing in maths Lab and course studies 7 studies (testing), 27 studies (spacing)
van Gog and Sweller 2015 Review of experiments on whether testing helps with complex material University, vocational and primary students; Netherlands and Australia 8 experiments
Karpicke and Aue 2015 Reply to van Gog and Sweller (abstract read)

3.3 Then complete solutions with steps missing

Section titled “3.3 Then complete solutions with steps missing”

Evidence: low. Small trials only, all pointing the same way.

Fading has no pooled estimate. The most-cited classroom result, near transfer 79% against 62% (Renkl, Atkinson and Maier 2000), compares two intact classes of 20 and 15 after one lesson (g ≈ 0.64, CI −0.05 to 1.32). A second experiment with undergraduates gave g ≈ 0.29. Fading adapted to each student beat fixed fading on delayed retention in Cognitive Tutor studies (Salden et al. 2010).

The studies.

Study What it was Who and where Students
Renkl, Atkinson and Maier 2000 Faded worked examples: a two-class quasi-experiment and a lab experiment Grade 9, lowest-track school, Germany (physics); US psychology undergraduates (probability) 35 (two classes); 54
Salden et al. 2010 Review of classroom studies adding worked examples to Cognitive Tutor software High school geometry, algebra and chemistry; US and Germany 4 geometry studies

3.4 Then practise: spaced out and mixed

Section titled “3.4 Then practise: spaced out and mixed”

Evidence: moderate. Reliable in classrooms, but smaller on real exams than on topic tests.

Spacing is reliable but smaller in classrooms. In the lab the best gap between study and review is about 20% of the time to the test for horizons of a few weeks, falling to about 5% at a year (Cepeda et al. 2008). In K-12 classrooms spacing gives g = 0.20 [0.03, 0.36], or 0.39 against fully massed practice and about zero against slightly less spaced practice (Ruitenburg et al. 2026). In nine real STEM courses the pooled gain was 2 percentage points, and the physics course went the wrong way (g = −0.12). In eight of those nine courses students saw only right or wrong, never the correct answer (Bego et al. 2024). An expanding schedule is no better than an even one (g = 0.03; Latimier, Peyre and Ramus 2021).

Mixed practice helps on targeted tests, less on real exams. Interleaving gives g = 0.40 in K-12 classrooms and 0.54 in maths (Ruitenburg et al. 2026); 0.34 in maths with a trim-and-fill overall estimate of 0.29 (Brunmair and Richter 2019). In a US Grade 7 trial, interleaved students scored 61% against 38% a month later (d = 0.83; Rohrer et al. 2020). Two findings hold it back for JEE:

  • In undergraduate physics, big gains on surprise tests did not reach the real midterms (d = 0.20 and 0.02; Samani and Pan 2021).
  • In a year-long Lagos trial, the term-test gain of 0.29 SD was not detected on the year-end test (0.04, CI about −0.27 to 0.35), with some sign of losses at the top (van der Haar et al. 2023).

With feedback after every item, interleaving showed no benefit (g = 0.03, 4 studies), against 0.82 with feedback at the end of the set (Ruitenburg et al. 2026). That rests on seven studies, so treat it as a hypothesis to test.

The studies.

Study What it was Who and where Students
Cepeda et al. 2008 Online experiment varying the gap between study sessions and the final test Adults learning obscure facts 1,354
Ruitenburg et al. 2026 See 3.2: spacing in real school classrooms
Bego et al. 2024 Questions randomly assigned to spaced retrieval within each student, nine courses Introductory STEM undergraduates, University of Louisville 578 in eight courses; 180 in calculus
Latimier, Peyre and Ramus 2021 Meta-analysis of spaced against massed retrieval Mostly undergraduates; 9 of 11 studies in the lab 11 studies, 39 effects
Brunmair and Richter 2019 Meta-analysis of mixed (interleaved) against blocked practice Mostly adults in the lab; paintings, photos, maths, texts 59 papers, 238 effects
Rohrer et al. 2020 Class-randomised trial of mixed maths homework Grade 7, Florida, 54 classes, 15 teachers 787
Samani and Pan 2021 Two lecture sections swapped mixed and blocked homework Introductory physics undergraduates, US About 290
van der Haar et al. 2023 Class-randomised trial of mixed maths practice over a full year Grade 5, low-cost private schools in Lagos, Nigeria, 58 schools 665 (term tests); 518 (end of year)

3.5 When wrong: the correct answer and the working, then the student’s own error

Section titled “3.5 When wrong: the correct answer and the working, then the student’s own error”

Evidence: moderate. Two meta-analyses of feedback types agree; self-explanation is well supported; explaining one’s own error is untested.

The often-quoted comparison, d = 0.99 for explanatory feedback against 0.46 for right-or-wrong (Wisniewski, Zierer and Hattie 2020), does not compare like with like. The 0.99 rests on 42 effect sizes, its category adds self-regulation advice rather than “why”, and its 0.46 category already includes the correct answer.

Two better sources compare feedback types directly:

So an explanation adds about 0.15 to 0.2 SD over the correct answer, and in maths transfer items (12 comparisons) the plain correct answer did better. Immediate and delayed feedback do equally well (g = 0.03; Kandemir et al. 2026, abstract only). A lesson the evening after a test loses nothing by waiting.

Self-explanation helps, mostly for concepts, and costs practice time. Prompts gave g = 0.55 overall, 0.43 in high school and 0.41 with time held equal (Bisra et al. 2018). The claim that multiple-choice prompts fail rests on 2 effect sizes. Fill-in-the-blank prompts, also constrained, scored 0.90 on another 2, and sentence-building menus worked in algebra classrooms (Booth et al. 2013). Adding prompts to worked examples was associated with a smaller effect in maths (Barbieri et al. 2023). One or two prompts at the key principle, with a menu of answers, is a defensible default.

Erroneous examples help a little, and only when the error is explained. The pooled effect is small (g = 0.14; Alemdag, Eichelmann and Narciss 2025, abstract only), larger on delayed tests (d = 0.33 at one week; McLaren, Adams and Mayer 2015). Every study used a fictitious student’s error. None tests showing a student their own wrong answer as the example, which is what the coach would do.

The studies.

Study What it was Who and where Students
Wisniewski, Zierer and Hattie 2020 Meta-analysis of feedback, by type of feedback All levels; mixed outcomes About 61,000 (959 effects)
Van der Kleij, Feskens and Eggen 2015 Meta-analysis of feedback on test items in computer-based learning (abstract read) Mostly university 40 studies, 70 effects
Mertens, Finn and Lindner 2022 Network meta-analysis of feedback types, randomised experiments only 58% undergraduates, 21% middle school; a third of effects in maths, a third in science About 89 students per effect
Kandemir et al. 2026 Meta-analysis of immediate against delayed feedback (abstract read) Mixed 51 studies, 160 effects
Bisra et al. 2018 Meta-analysis of prompting learners to explain to themselves Primary to professional; mostly North America and Europe 5,917 in 64 reports; 1,415 in high school
Booth et al. 2013 Two classroom trials of correct and incorrect worked examples in algebra software High and middle school Algebra I, US About 115 and 67
Barbieri et al. 2023 See 3.2
Alemdag, Eichelmann and Narciss 2025 Meta-analysis of learning from erroneous examples (abstract only) Not read 42 papers, 177 effects
McLaren, Adams and Mayer 2015 Trial of erroneous examples in a web tutor, with a delayed test Grade 6 decimals, Pittsburgh 390

3.6 Retry a new question of the same kind, without help

Section titled “3.6 Retry a new question of the same kind, without help”

Evidence: low to moderate. A re-analysis of the classic tutoring studies, not a trial of the retry alone.

About half of the “two sigma” tutoring effect comes from the retest: a quiz, feedback and correction, then a second quiz with new questions (von Hippel 2024).

The studies.

Study What it was Who and where Students
von Hippel 2024 Re-analysis of the two small trials behind Bloom’s “two sigma” claim Grades 4, 5 and 8, US; probability and map-reading Two small dissertation trials

3.7 Strong in the chapter: straight to problems

Section titled “3.7 Strong in the chapter: straight to problems”

Evidence: moderate. One meta-analysis; the cost is smaller when prior knowledge is measured by a test.

Expertise reversal is real and lopsided. The first meta-analysis puts more help at +0.51 for students with little prior knowledge and −0.43 for knowledgeable ones (Tetzlaff et al. 2025). When prior knowledge is measured by a test, as a coach would measure it, the cost for strong students is about −0.21. Withholding an example from a weak student costs more than giving an unneeded one to a strong student, so when unsure, show it. A one-question “what is your first step?” probe sorted students well in the developers’ own studies (correlation up to .92 with longer tests; Kalyuga 2007).

The studies.

Study What it was Who and where Students
Tetzlaff et al. 2025 Meta-analysis of the expertise reversal effect Primary to adult, 18 secondary-school studies; 38 of 60 studies in STEM 5,924 in 60 studies
Kalyuga 2007 Review of expertise reversal experiments High school students in Australia and engineering students in the US, in the studies it tabulates 42 to 124 per experiment

3.8 AI tutor: confirm the wrong idea, then refute it

Section titled “3.8 AI tutor: confirm the wrong idea, then refute it”

Evidence: moderate for refutation, mostly from reading studies with university students. Several studies show that one student’s answer to one question is a noisy signal.

Refutation works, at a modest size. Naming a misconception and showing where it fails beats a standard explanation: g = 0.41 [0.30, 0.51], I² = 61%, and 0.28 in a trim-and-fill check (Schroeder and Kucera 2022). Most of the studies are reading texts for university students, and only 4 are in secondary school. Telling students “don’t make this mistake” does not work (McDermott 2001).

A wrong option is a class-level signal, not a diagnosis of one student.

  • Systematic at class level. Misconception distractors peak in appeal for middle-ability students (Sadler 1998). Teachers who knew the popular wrong answer saw larger gains, though only among higher-achieving students and in an observational design (Sadler and Sonnert 2016).
  • Noisy for one student.

Contrasting cases have promise, on thin evidence. Comparing cases gives d = 0.60 overall but 0.34 [−0.01, 0.68] for adolescents, from 4 studies (Alfieri, Nokes and Schunn 2011). Two well-known studies:

  • Schwartz and Bransford 1998, about 12 college students per arm: those who analysed cases and then heard a lecture made 43.8% of possible predictions a week later, against 14.6% for a summary then the lecture.
  • Schwartz et al. 2011, Grade 8 physics: inventing with contrasting cases then being told gave 53.5% against 23.0% on transfer at 21 days.

The commit step has no evidence of its own. Peer Instruction roughly doubled the conceptual gain at Harvard (normalised gain 0.25 to 0.49; Crouch and Mazur 2001). That compares one traditional year with the next and tests a whole package: vote, discuss with a neighbour, revote, explanation. No study isolates the commit step (Vickrey et al. 2015). The evidence points to peer discussion plus explanation together (Smith et al. 2011), and a one-to-one coach has no peers. Problem-solving scores rose too.

The studies.

Study What it was Who and where Students
Schroeder and Kucera 2022 Meta-analysis of refutation texts Mostly post-secondary (30 of 44 comparisons); mostly science; Europe and North America 3,869
McDermott 2001 Synthesis of a physics education research programme (Oersted lecture) University physics, US
Sadler 1998 Analysis of answers to a test whose wrong options are common misconceptions Grades 8 to 12 astronomy, US 1,250
Sadler and Sonnert 2016 Teachers’ knowledge of misconceptions and their students’ gains Grades 7 and 8 physical science, 181 teachers, US 9,556
Lasry et al. 2011 The same physics test taken twice, with no teaching in between College electricity and magnetism 100
Wells et al. 2019 Analysis of which wrong answers go together on a physics test University physics, US 9,225 tests (before and after teaching)
Kaltakci Gurel, Eryilmaz and McDermott 2015 Review of tests used to diagnose misconceptions 273 articles
Alfieri, Nokes and Schunn 2011 Meta-analysis of learning by comparing cases Mostly adults (41 of 57 experiments); US and Europe 57 experiments
Schwartz and Bransford 1998 Classroom experiment: analyse contrasting cases before a lecture College students new to psychology, US 36
Schwartz et al. 2011 Classroom experiment: invent a formula from contrasting cases, then be taught Grade 8 physics, four classes, US 128
Crouch and Mazur 2001 Ten years of Peer Instruction compared with earlier traditional years Harvard introductory physics 298 in the 1990-91 comparison
Vickrey et al. 2015 Review of Peer Instruction research Undergraduate STEM, mostly US
Smith et al. 2011 Within-course experiment: peer discussion, instructor explanation, or both Undergraduate genetics, US One course (numbers not extracted)

3.9 AI tutor: hold back the answer and set the retry

Section titled “3.9 AI tutor: hold back the answer and set the retry”

Evidence: low. Seven school trials pool to about 0.16 SD with a wide spread; one large trial shows an answer-giving chatbot does harm.

Structured AI tutors, pooled here. Seven school trials measured learning without the AI. Together they give 0.16 SD [0.05, 0.28], but a new setting could land anywhere from −0.21 to 0.54.

Pool Trials Estimate 95% CI HKSJ interval I² 95% prediction interval
All structured tutors 7 0.16 0.05 to 0.28 0.01 to 0.31 81% −0.21 to 0.54
Independent or standardised tests only 4 0.11 0.01 to 0.21 −0.07 to 0.29 58% −0.29 to 0.51
Excluding the two high-risk trials 5 0.11 −0.01 to 0.22 −0.08 to 0.30 78% −0.29 to 0.51

The seven trials:

Trial Country Result
Bastani et al. 2025, guarded tutor Turkey −0.01 SD
De Simone et al. 2025 Nigeria 0.31
Henkel et al. 2024 Ghana 0.36, with an 11-school cluster correction
LearnLM Team, Google and Fab AI 2026 Sierra Leone 0.26 on an independently built test, time-matched
Oreopoulos and Low 2026, Khanmigo US 0.07 a year
Harrison et al. 2026 UK 0.33
LearnLM Team, Google and Eedi 2026 UK −0.01 on Star Maths, a standardised test; every AI message approved by a tutor

Other estimates line up:

  • Dang and Do 2026: 0.21, or 0.16 without the two weakest trials.
  • Bartoš et al. 2026: a not-yet-reviewed re-analysis of 67 AI-in-education meta-analyses. It cuts the average from 0.63 to 0.20 after correcting for publication bias, and puts AI used as a tutor at about zero.
  • Retracted: one widely cited 2025 meta-analysis (g = 0.87) has been withdrawn.

Unstructured AI can harm.

  • Bastani et al. 2025, Turkey. A ChatGPT-like assistant raised practice scores but lowered the unaided exam minutes later by 0.19 SD (my conversion of −0.054 on a 0 to 1 scale). The guarded tutor raised practice scores by 1.26 SD and left the exam unchanged (−0.01 SD).
  • Liu et al. 2026. In a US university trial, access to a course-integrated AI tutor lowered grades by 0.27 to 0.37 SD.

The gain has to come from what students do unaided, not from the AI’s explanations.

The studies.

Study What it was Who and where Students
Bastani et al. 2025 Class-randomised trial of GPT-4 maths tutors, one plain and one guarded Grades 9 to 11, one high school in Turkey, about 50 classes About 1,000
De Simone et al. 2025 Trial of an after-school GPT-4 English programme among volunteers Senior Secondary 1 (about 15), nine public schools, Benin City, Nigeria 759 analysed of 1,328
Henkel et al. 2024 School-randomised trial of Rori, a WhatsApp maths tutor used one hour a week in study hall Grades 3 to 9, Rising Academies schools, Ghana, 11 schools 477 of 637
LearnLM Team, Google and Fab AI 2026 Classroom-randomised trial of Gemini Guided Learning Grades 7 to 8, 48 classrooms in 12 government-supported schools, Sierra Leone 1,763
Oreopoulos and Low 2026 Grade-randomised trial of Khanmigo in remedial maths periods over two years Middle school, Tennessee, 53 clusters 6,902 student-terms
Harrison et al. 2026 Micro-randomised trials of an AI tutor (Medly) in GCSE science Years 10 to 11, English secondary schools 644 of 929
LearnLM Team, Google and Eedi 2026 Student-randomised four-arm trial of Gemini tutoring on Eedi Years 8 to 10, 10 UK state schools, 58 classes 1,179
Dang and Do 2026 Review pooling eight AI tutoring comparisons Ghana, Nigeria, Sierra Leone, Turkey, US, Germany, Italy 8 comparisons
Bartoš et al. 2026 Meta-meta-analysis of AI and learning (preprint) All levels 1,840 effects
Liu et al. 2026 Instructor-randomised trial of course-integrated AI tutoring Undergraduates, a large US public university, 30 instructors 2,379

3.10 Judge the retry later, on new questions, unaided

Section titled “3.10 Judge the retry later, on new questions, unaided”

Evidence: moderate. Two large trials, and a consistent gap between researcher-written and independent tests.

Practice scores mislead. With a guarded AI tutor, practice scores rose 1.26 SD while the exam did not move (Bastani et al. 2025). Students also rate active lessons lower even when they learn more (Deslauriers et al. 2019).

  • The same gap in a trial built like the coach. In 10 UK schools over 10 weeks (1,179 students), a Gemini tutor whose every message was approved by an expert tutor took students from 43% to 57% on correcting the mistake in front of them, and from 48% to 60% on the same question a week or more later. On Star Maths, an external standardised test, it moved scores by −0.01 SD [−0.15, 0.14] (LearnLM Team, Google and Eedi 2026).
  • Depth costs breadth. Each tutoring chat took 4 to 7 minutes, so tutored students finished 99 units against 132 and met fewer topics. Each extra standard deviation of units completed was worth 0.09 SD on Star Maths. A coach lesson spends exam-prep time; it has to be worth more than the questions it displaces.
  • Girls gained, boys did not. Under standard support boys pulled ahead of girls by 0.31 SD over the trial; under tutoring the gap was near zero. Two-thirds of the sample were boys, one school all-boys. Adjusting for coverage and gender, the AI arm sits at +0.05 [−0.10, 0.20]. This is exploratory, and the authors say so.
  • Who wrote the test. Effects on tests written by the researchers run 0.24 to 0.32 SD higher than on independent tests in the same studies (Wolf and Harbatkin 2023, as reported by Gilbert and Soland 2026). Kulik and Fletcher’s 0.72 against 0.08 is the extreme case. As a rule of thumb, halve a researcher-test effect, and expect less still on JEE.

The studies.

Study What it was Who and where Students
Deslauriers et al. 2019 Crossover trial: active class against lecture, with students’ sense of how much they learned Harvard introductory physics 149
Gilbert and Soland 2026 Item-level meta-analysis of researcher-written against independent tests Trials with item data, mostly US 30 studies, 45 effects
Bastani et al. 2025 See 3.9
LearnLM Team, Google and Eedi 2026 See 3.9

3.11 How big the effects are

Section titled “3.11 How big the effects are”

Evidence: low. Planning ranges are judgements, and no study here was run with JEE students.

The planning ranges are set out in the table at the top of this section. For scale, these are what whole tutoring programmes achieve on standardised tests.

Older tutoring software. Step-based tutors beat no tutoring by d = 0.76, mostly in short lab studies with college students (VanLehn 2011). His 0.31 for answer-only systems comes from an older, different pool of computer-based instruction studies that he imports, not from his own comparisons; the two direct comparisons of step-based and answer-only tutors give 0.40. On standardised tests and at school scale:

Source Estimate
Kulik and Fletcher 2012 0.08, against 0.72 on locally written tests
Leite et al. 2025 0.20
Roschelle et al. 2016 0.18
Pane et al. 2014 0.12 to 0.21 in year 2, null in year 1

An Andes physics tutor gain of 0.25 on a multiple-choice final was not significant for science and engineering majors (VanLehn et al. 2005).

Adaptive practice software in India and other poorer countries.

Human tutoring at scale gives 0.16 to 0.21 SD on standardised tests (Kraft, Schueler and Falken 2024), down from 0.37 across all trials (Nickow, Oreopoulos and Quan 2020). An AI tutor as good as a good human tutor should be expected here, not at Bloom’s two sigma.

What moves the size of an effect.

  • Who wrote the test. Effects on tests written by the researchers run 0.24 to 0.32 SD higher than on independent tests in the same studies (Wolf and Harbatkin 2023, as reported by Gilbert and Soland 2026). Kulik and Fletcher’s 0.72 against 0.08 is the extreme case. As a rule of thumb, halve a researcher-test effect, and expect less still on JEE.
  • Study size and length. Tutoring studies with 80 or fewer students average 0.76; those with over 250 average 0.30 (Kulik and Fletcher 2012). Human tutoring falls from 0.42 to 0.16 at 1,000 or more students (Kraft, Schueler and Falken 2024).
  • Prior knowledge. It reverses worked examples (section 3.7) and decides whether a wrong answer is a misconception or a guess (section 3.8). Results also split by level in the AI trials: stronger students gained more in Sierra Leone and Nigeria, weaker ones in the US maths trial (Roschelle et al. 2016).
  • Lab against classroom. Every technique with both shrinks in the classroom: testing (lab 0.50, classroom procedures −0.12), spacing (lab 0.7 to 1.0, classroom 0.20), worked examples (pooled 0.48, largest classroom trial 0.10).
  • Publication bias. Severe in AI-in-education studies (a factor of about three; Bartoš et al. 2026). Checked only by fail-safe N in several older meta-analyses (Adesope; Bisra), so their published means are upper bounds.
  • Population. About 6% of classroom retrieval-practice effects come from outside rich Western countries (Agarwal, Nunes and Blunt 2021). None of the 181 estimates is from Indian students preparing for an entrance exam.

The studies.

Study What it was Who and where Students
VanLehn 2011 Review of experiments comparing human tutors, step-based and answer-based software Mostly US college students, many short lab studies 28 comparisons
Kulik and Fletcher 2012 Meta-analysis of intelligent tutoring system evaluations US school, college and military 45 evaluations
Leite et al. 2025 Meta-analysis of intelligent tutors in US schools, to What Works Clearinghouse standards US K-12 maths and reading 18 studies, 77 effects
Roschelle et al. 2016 School-randomised trial of ASSISTments maths homework Grade 7, 43 schools, Maine 2,850
Pane et al. 2014 School-randomised trial of Cognitive Tutor Algebra I at scale High school, 147 schools in seven states 11,066
VanLehn et al. 2005 Quasi-experiment with the Andes physics tutor over five semesters US Naval Academy introductory physics 628 with Andes, 438 without
Major, Francis and Tsapali 2021 Meta-analysis of randomised trials of technology-supported personalised learning in poorer countries School-age, five countries, many in China 53,029 in 16 trials
Muralidharan, Singh and Ganimian 2019 Lottery trial of Mindspark after-school centres Grades 6 to 9, Delhi 619
Muralidharan and Singh 2025 School-randomised trial of Mindspark labs during the school day Grades 1 to 8, government schools, Rajasthan, 80 schools 8,733
Oreopoulos, Keyes-Krysakowski and Agarwal 2026 School-randomised trial of supervised Khan Academy practice Grades 6 to 8, residential government schools, Uttar Pradesh, 74 schools analysed 7,888
Banerjee et al. 2007 School-randomised trial of computer-assisted maths Grade 4, municipal schools, Vadodara About 4,700
Kraft, Schueler and Falken 2024 Meta-analysis of tutoring trials, by programme size PreK-12, mostly US 265 trials
Nickow, Oreopoulos and Quan 2020 Meta-analysis of PreK-12 tutoring trials PreK-12, mostly US and UK 96 studies, 732 effects
Agarwal, Nunes and Blunt 2021 Systematic review of retrieval practice in classrooms K-12, undergraduate and medical school 5,374 in 50 experiments
Gilbert and Soland 2026 See 3.10
Bartoš et al. 2026 See 3.9

3.12 Other: negative marking and the attempt rule

Section titled “3.12 Other: negative marking and the attempt rule”

Evidence: the arithmetic is certain; the benefit of teaching it is unmeasured.

The arithmetic. Under +4/−1, answering is worth 5p − 1 marks in expectation, where p is the chance of being right. It pays whenever p is above 20%:

Guess Expected marks
Blind, four-option MCQ +0.25
After ruling out one option +0.67
After ruling out two +1.5
Blind, numerical question about −1

The JEE Main 2027 rules confirm the scheme, and break ties partly on the ratio of wrong to right attempts (National Testing Agency 2026).

Who skips, and what it costs.

  • Women skip more. In a lab, women skipped 2.84 of 20 questions against 1.57 for men, with no gap in knowledge (Baldiga 2014).
  • The score cost is small. When Chile dropped its penalty, the gender gap in skipped questions fell 70% but the score gap fell only 0.025 SD (Coffman and Klinowski 2020). In Turkish entrance-exam data, women’s answer threshold was 0.26 against men’s 0.25, and the authors favour keeping negative marking (Akyol, Key and Krishna 2016).
  • The counter-evidence is weak. A study finding no gender gap in risk aversion has 83 students at one school (Krishna, Akyol and Ozer 2025).

JEE makes a blind MCQ guess worth +0.25, more generous than any exam studied, so skipping costs more here.

No trial shows that teaching the rule raises scores. Test preparation that included test-taking skills gave g = 0.35 against 0.05 without, but that comparison is across studies and confounded (Hao et al. 2026). The rule also depends on the student’s own p. On misconception questions students rule out the right answer with confidence. And every extra guess makes wrong answers less informative for diagnosis.

The studies.

Study What it was Who and where Students
National Testing Agency 2026 JEE (Main) 2027 Information Bulletin: the marking rules JEE Main candidates
Baldiga 2014 Lab experiment: SAT-style questions with and without a penalty for wrong answers Harvard Business School lab pool, US 406
Coffman and Klinowski 2020 Natural experiment: Chile dropped its penalty for wrong answers in 2015 All university entrance test takers, 2013 to 2016 About 3 million test records
Akyol, Key and Krishna 2016 Model of guessing fitted to exam records Turkish university entrance exam takers Administrative data
Krishna, Akyol and Ozer 2025 Experiment varying the time limit on a penalised test Grade 12, one high school in Turkey 83
Hao et al. 2026 Meta-analysis of test preparation for large-scale tests School and university students About 16,700 in 28 studies

3.13 Video, live teaching and self-study

Section titled “3.13 Video, live teaching and self-study”

We searched this separately on 2 November and read 46 sources: 28 mostly from universities in richer countries, 18 from schools in poorer ones. None is on JEE or NEET preparation.

  • The medium on its own changes little. Across 318 comparisons, distance teaching matched the classroom (g = 0.01; Bernard et al. 2004). Swapping video for existing teaching gives a small gain, g = 0.28: smallest when it replaces a lecturer, larger when it replaces text (Noetel et al. 2021). Both reviews draw mostly on college and adult courses, so there is no strong evidence either way for school students learning ahead of grade.
  • Adding video beats replacing the teacher. Video on top of existing teaching gives g = 0.80 in college courses (Noetel et al. 2021). In Pakistan, short videos the teacher played in class raised Grade 8 maths and science by 0.30 SD and the government exam by 0.22 SD. Similar content on Grade 6 students’ own tablets lowered scores by 0.42 SD (Beg et al. 2022). Both trials taught grade-level content; none we found tests acceleration or Grades 11–12.
  • Weaker students lose most when live teaching is replaced. At West Point, online classes lowered final grades by 0.215 SD, driven by students below the median (Kofoed et al. 2024). Two other randomised trials find the same pattern: recorded or streamed lectures hurt weaker students and helped or did not hurt stronger ones (Figlio, Rush and Yin 2013; Cacault et al. 2021).
  • What happens in the lesson matters more than the medium. Active lessons beat lectures by 0.47 SD in undergraduate STEM (Freeman et al. 2014) and cut the exam gap for low-income and minority students by a third (Theobald et al. 2020).
  • Broadcast lessons help where there was little teaching. Live satellite lessons in Ghana’s primary schools raised maths by 0.23 SD (Johnston and Ksoll 2017).
  • Avanti’s own video programmes have not moved independent exams. In Haryana’s Sankalp trial (240 schools), classroom videos with workbooks lowered maths by about 0.15 SD, and observed teaching quality fell by 0.46 SD (de Barros 2022). In Chennai, the blended programme raised basic skills but not Class X board scores (−0.28 SD, p = 0.06; Chiplunkar, Dhar and Nagesh 2020). In Tanzania, videos working through past O-level papers moved the national exam by no more than 0.13 SD (Seo 2017).

For Avanti, recorded classes work best as something the teacher runs, not a replacement for the teacher, and the students most at risk when live teaching is replaced are the weaker ones Avanti serves.

The studies.

Study What it was Who and where Students
Bernard et al. 2004 Meta-analysis of distance against classroom teaching, 1985 to 2002 K-12 to adult, mostly North America 54,775 (318 outcomes)
Noetel et al. 2021 Meta-analysis of randomised trials of video in higher education University, mostly health sciences 7,776 in 83 studies
Beg et al. 2022 School-randomised trial: videos played by the teacher in class, or on students’ own tablets Grade 8 (teacher-led) and Grade 6 (tablets), government schools, Punjab, Pakistan 2,622 in 59 schools; 3,058 in 71 schools
Kofoed et al. 2024 Trial: the same instructor online or in person West Point cadets, introductory economics 551
Figlio, Rush and Yin 2013 Trial: live lectures or the same lectures online Introductory microeconomics, a large US university 296
Cacault et al. 2021 Trial: access to live-streamed lectures First-year economics and management, University of Geneva 23,766 student-course-weeks
Freeman et al. 2014 Meta-analysis of active learning against lecturing Undergraduate STEM, mostly US 158 studies
Theobald et al. 2020 Pooled student-level data on active learning and achievement gaps Undergraduate STEM, US 9,238 in 15 studies
Johnston and Ksoll 2017 School-randomised trial of live satellite lessons Grades 2 to 5, rural primary schools, Ghana, 147 schools 12,848 student-rounds
de Barros 2022 Three-arm school-randomised trial of Avanti’s Sankalp classroom videos and workbooks Grades 9 to 10, government schools, Haryana, 240 schools 24,584
Chiplunkar, Dhar and Nagesh 2020 School-randomised trial of Avanti’s blended programme Grade 9, Chennai Corporation schools, 24 schools 756
Seo 2017 Six-arm school-randomised trial of exam-preparation videos Grade 11 (O-level), Tanzania, 164 schools 10,171

4. Discussion

Section titled “4. Discussion”

4.1 Implications for the AI coach

Section titled “4.1 Implications for the AI coach”
  1. Show the solution first on multi-step problems; test on the idea. For a wrong JEE-style answer: worked solution, then a new question of the same kind. Keep recall testing for formulas, definitions and one-idea checks.
  2. Every piece of feedback gives the correct answer and the working. Right or wrong alone is close to worthless. The explanation of why the chosen option is tempting is worth about 0.2 SD on top. Test it against answer-plus-working alone.
  3. Route by level in the chapter, and when unsure, show the example. A one-question probe (“what is your first step?”) decides. Strong students go problem-first and see the solution on request; a miss on the probe drops them to the worked route.
  4. Confirm the presumed misconception before teaching to it. Use the lesson’s opening question plus a three-point confidence rating. High confidence on the mapped option means misconception, so refute it. Low confidence or a “guessed” flag means a gap, so go to the worked example. For a student scoring near chance in the chapter, skip refutation and teach explicitly.
  5. Measure on new questions, later, without help. Lesson accuracy and student ratings mislead: practice scores rose 1.26 SD with a guarded AI tutor while the exam did not move (Bastani et al. 2025). Students also rate active lessons lower even when they learn more (Deslauriers et al. 2019).
  6. Use an even spacing schedule and mixed retest sets, with feedback at the end of each set. Expect about 0.2 on retested ideas, not the lab’s 0.7.
  7. Teach the attempt rule as an experiment. Eliminate on reasons, not familiarity, and never guess blind on a numerical question. Before targeting by gender, check Avanti’s own data for skip rates within ability bands. Randomise the nudge across classes.
  8. Plan for small numbers on JEE.
    • Coached chapters: plausibly 0.3 to 0.6 SD on new questions, if the loop runs.
    • JEE Main: about 0.05 to 0.15 SD, with a real chance of zero.
    • Sample size: detecting 0.10 SD on JEE needs about 940 students per arm. The 52-student pilot can measure only the coached chapters.

4.2 Limitations

Section titled “4.2 Limitations”
  • No study here is on Indian students preparing for JEE or NEET. The closest are Grade 9 in a German low-track school (one lesson), US Algebra I and chemistry classrooms, Grades 6 to 8 in Uttar Pradesh residential schools, and school AI trials in Nigeria, Ghana and Sierra Leone. Three Indian AI-tutor trials are registered; none has reported.
  • The planning ranges are judgements. They are drawn from the evidence and say what they rest on, but another reviewer could set them differently. Only the AI-tutor pool is a formal pool, and it has seven trials.
  • Some sources were read as abstracts only, and several were behind publisher bot checks and not opened. Their figures are marked or left out.
  • Effects do not add up. Each technique was tested against a comparison that already had some of the others.
  • The reading of why students chose each wrong answer in the worked walkthrough (The AI coach, step by step) is ours. We have their answers, not their working.

5. Conclusion

Section titled “5. Conclusion”

The evidence supports a short loop after each test: the correct answer and the working, then the student’s own error, solved examples for students weak in the chapter, and a new question tried unaided days later. Each step has moderate or better evidence on the coached material. AI adds most where a step is hard to do by hand at scale, naming the wrong idea behind an answer and setting the retry, and does harm when it hands out answers. The gains on a full JEE paper will be small, so the first pilot should be judged on new questions in coached chapters. A test on JEE Main needs about 940 students per arm, a trial across several centres.

References

Section titled “References”

Every work below was downloaded and read in an archived copy between 26 October and 1 November 2026, except those marked abstract only. The download links open for avantifellows.org accounts only.

Appendix A. Data and code

Section titled “Appendix A. Data and code”
  • meta/effects.tsv: every extracted estimate, 181 rows, with source, design, CI, independence and the archived file.
  • meta/headline.tsv: the planning range and certainty per technique, with what each rests on.
  • meta/pool.py: the random-effects pools and the figure. python3 analysis/ai-coach/meta/pool.py.
  • meta/check.py: asserts that every planning range and pooled figure quoted in this paper matches the two files above.

Appendix B. Pressure test: classroom technology against self-study and tutoring

Section titled “Appendix B. Pressure test: classroom technology against self-study and tutoring”

The companion note, What works, makes two claims. We checked each one against the sources in this review.

Claim 1: the evidence on technology improving classroom instruction is sparse and the effects are small. This mostly holds, with one correction. The evidence is not sparse, it is mixed, and what decides the result is who runs the technology.

  • Replacing the teacher does little or does harm. Across 318 comparisons, distance teaching matched the classroom (g = 0.01; Bernard et al. 2004). Avanti’s own classroom videos in Haryana lowered maths by about 0.15 SD (de Barros 2022). The Chennai blended programme did not move Class X board scores (Chiplunkar, Dhar and Nagesh 2020).
  • Technology the teacher runs can help. In Pakistan, short videos that teachers played in class raised the government exam by 0.22 SD. Similar content on students’ own tablets lowered scores by 0.42 SD (Beg et al. 2022).
  • None of these trials tests learning ahead of grade, or Grades 11–12. On JEE preparation the classroom evidence says little either way.

Claim 2: self-study and tutoring have large impacts if implemented well. This holds for human tutoring. It is weaker for software, and for self-study on multi-step problems. “If implemented well” carries most of the weight.

  • Human tutoring is the strongest evidence in the review. It averages 0.37 SD across all trials (Nickow, Oreopoulos and Quan 2020). On standardised tests at scale it gives 0.16 to 0.21 SD (Kraft, Schueler and Falken 2024). Kraft’s benchmarks call 0.05 to 0.20 SD medium and 0.20 or more large for a school intervention (Kraft 2020).
  • The effects shrink with scale. Human tutoring falls from 0.42 SD to 0.16 SD in programmes of 1,000 or more students (Kraft, Schueler and Falken 2024).
  • Software gives less. Across seven K-12 trials, AI tutors pool to 0.16 SD, and to 0.11 SD on independent tests (section 3.11). The largest recent trial, LearnLM with Eedi in the UK, found −0.01 SD on Star Maths (LearnLM Team, Google and Eedi 2026).
  • Adaptive self-study software fills gaps but does not move the grade-level exam. Mindspark raised Delhi students’ maths by 0.37 SD on below-grade items, with no gain on grade-level school exams (Muralidharan, Singh and Ganimian 2019).
  • Self-study techniques depend on the task. Practice testing on facts and formulas gives about 0.45 SD. On multi-step problems the estimate is 0.10 SD, and its range crosses zero (section 3, planning ranges).

What this means for the companion note. The direction holds: put the coach in self-study and tutoring, not in the lecture. Two words overstate it. “Sparse” is closer to “mixed, and small or negative where technology replaces the teacher”. “Large” fits human tutoring at 0.16 to 0.21 SD. Plan software and AI tutoring at about 0.1 SD on independent exams. The strongest classroom result, video that the teacher runs, matches what the coach proposes: the teacher stays in the session.