AI Coach Reading Library

Research the plan’s fleet downloaded and read but did not cite in the first AI coach plan, since retired: 26 papers, each in plain language. Who was studied, how, what it found, why it matters for our students, and why it is not in the plan. Download links open the archived copy and need an Avanti Google login. Web pages, terms of service, exam papers and second copies of cited papers are left out.

AI tutors

Little Impact of ChatGPT on High School Test Scores. Did everyday student ChatGPT use change US test scores?

Huntington-Klein 2026, arXiv working paper. Download · Source

Who was studied: US high-school students in aggregate: state and metro-area test scores across the US, compared with ChatGPT search activity in 210 metro areas, 2023-2025.
How: Observational. Areas where ChatGPT activity drops most in the summer holidays are treated as areas with heavy school use of ChatGPT, and their test-score changes are compared with other areas.
What it found: No clear effect. One standard deviation more school-related ChatGPT use changed test scores by -0.0096 SD (standard deviations, the usual unit for comparing test-score gains), not statistically different from zero; the author bounds any harm at about 0.04 SD.
Why it matters for us: Unsupervised AI use has not shown a large average harm on test scores, but averages can hide harm in students who let the AI do practice for them.
Why it is not in the plan: weak identification; background only

Your Brain on ChatGPT. Brain activity and recall when adults write essays with AI.

Kosmyna et al. 2025, arXiv preprint (not peer reviewed), MIT Media Lab. Download · Source

Who was studied: 54 adults aged 18-39 from five Boston-area universities; 18 of them returned for a fourth session. Task: essay writing.
How: Lab experiment. Participants were assigned to write essays with ChatGPT, with a search engine, or with no tool, over three sessions, with brain activity recorded by EEG (electrodes on the scalp); in session 4 some groups switched tools.
What it found: The ChatGPT group showed the weakest brain connectivity and felt less ownership of their essays. In session 1, 83% (15 of 18) of the ChatGPT group could not correctly quote their own essay minutes after writing it, against 11% in the other two groups.
Why it matters for us: If the coach writes or solves for the student, the student may not remember or own the work; the coach should make the student do the thinking.
Why it is not in the plan: small adult lab study, not peer reviewed

Tutoring and coaching

What Impacts Should We Expect from Tutoring at Scale? How tutoring gains shrink as programmes grow.

Kraft, Schueler and Falken 2024, EdWorkingPaper 24-1031 (Brown University). Download · Source

Who was studied: 265 randomised trials of tutoring programmes for US school students (282 in the October 2024 cover abstract), maths and reading, all grades.
How: Meta-analysis (a statistical pooling of many studies’ results), with the results broken down by how many students each programme served.
What it found: Across all trials tutoring raised scores by 0.42 SD (standard deviations, the usual unit for comparing test-score gains). But the gain falls with size: 0.55 SD for programmes under 100 students, 0.21 SD for 400-999 students and 0.16 SD for 1,000 or more. High-school maths showed 0.55 SD, from only a few studies.
Why it matters for us: Even strong human tutoring delivers about 0.16-0.21 SD at scale, so an AI coach used a few hours a week should be planned against a smaller target.
Why it is not in the plan: background only; the plan sets its targets from Kraft 2019 and Evans and Yuan 2020

Designing for Human-AI Complementarity in K-12 Education. Smart glasses that show teachers which students need help.

Holstein and Aleven 2021, arXiv essay. Download · Source

Who was studied: US middle-school maths classes using an AI tutor for equation solving: design pilots with 14 teachers and 304 students, and a classroom trial with 343 students in 18 classes taught by 8 teachers.
How: Design account of a tool called Lumilo, plus a summary of a classroom experiment in which classes were randomly assigned to glasses with live analytics, glasses without analytics, or no glasses.
What it found: Students learned more when teachers wore the glasses with live analytics from the AI tutor. Teachers spent about 24 seconds per student visit, often prompting reflection or encouraging rather than reteaching content.
Why it matters for us: A teacher acting on short alerts from the coach is a realistic division of labour in a CoE classroom.
Why it is not in the plan: duplicate of a cited finding (the plan cites the trial itself, Holstein et al. 2018)

The Effects of Student Coaching in College. Phone and email coaching to keep college students enrolled.

Bettinger and Baker 2011, NBER Working Paper 16881. Download · Source

Who was studied: 13,555 US college students, mostly non-traditional (older or part-time), at eight public, private and for-profit institutions, 2003-04 and 2007-08.
How: Randomised controlled trial (RCT: students assigned by lottery). Coached students were contacted regularly by professional coaches from InsideTrack about goals, time management and study skills.
What it found: Coached students were 5.2 percentage points more likely to still be enrolled after 6 months (63.2% vs 58.0%) and 3.4 points more likely after 24 months.
Why it matters for us: Proactive contact about goals and study habits helps students persist, but it was measured on staying enrolled, not on exam scores.
Why it is not in the plan: population too far from ours; outcome is persistence, not scores

Student Coaching: How Far Can Technology Go? Texts and online exercises versus a personal coach.

Oreopoulos and Petronijevic 2016, NBER Working Paper 22630. Download · Source

Who was studied: Over 4,000 first-year undergraduates at a University of Toronto campus, Canada; the personal-coaching arm had only 24 students.
How: RCT with three arms: a one-time online goal-setting exercise, a text-message campaign of advice and motivation, or a personal coach (an older undergraduate).
What it found: Coached students gained 0.35 SD (standard deviations, the usual unit for comparing test-score gains) in GPA and 0.3 SD in average grades. The online exercise and the text messages had no effect on any academic outcome.
Why it matters for us: Generic messages do not move grades; two-way contact that remembers the student’s own situation might, which is what a coach must imitate.
Why it is not in the plan: coaching arm of 24 students is too small to rely on

Why Do College Going Interventions Work? Weekly mentoring for students unsure about applying to college.

Carrell and Sacerdote 2013, NBER Working Paper 19031. Download · Source

Who was studied: Grade 12 students at about 20 New Hampshire (US) high schools, nominated by counsellors as on the margin of applying to college; mentors were Dartmouth College students.
How: RCT comparing in-person weekly mentoring through the application season against information-only and cash-incentive arms.
What it found: For girls, mentoring raised college-going by 15 percentage points (about 30 points among those who took it up, against a 43% base). Effects for boys were much smaller, and information or money alone did nothing.
Why it matters for us: A human who stands in for missing adult support can change decisions, which suggests the coach should keep a named adult in the loop for Grade 12 choices.
Why it is not in the plan: outcome is college-going, not exam scores

The 2 Sigma Problem. The origin of the claim that tutoring adds two standard deviations.

Bloom 1984, Educational Researcher 13(6). Download · Source

Who was studied: US school students in grades 4, 5 and 8 in two doctoral dissertations, learning probability or cartography (map-making) over 11 class periods.
How: Essay summarising small randomised studies that compared a normal class, a mastery-learning class (test, correct, retest) and one-to-one tutoring.
What it found: The average tutored student scored about 2 standard deviations above the normal class (above 98% of it), and the average mastery-learning student about 1 standard deviation above (above 84%).
Why it matters for us: This is the source of the ‘2 sigma’ promise, and the coach should not be sold on it; the part worth keeping is retesting after feedback.
Why it is not in the plan: superseded by von Hippel 2024’s re-reading, which the plan cites

Adaptive platforms

Intelligent Tutoring Systems and Learning Outcomes: A Meta-Analysis. How much tutoring software helps, against what.

Ma, Adesope, Nesbit and Liu 2014, Journal of Educational Psychology 106. Download · Source

Who was studied: 107 effect sizes from studies with 14,321 learners at all levels of education, many subjects and countries.
How: Meta-analysis comparing students using intelligent tutoring systems (software that models what a student knows and adapts) with other forms of teaching.
What it found: Tutoring software beat large-group teacher-led classes (g = 0.42, an effect size in standard deviations), other computer instruction (0.57) and textbooks (0.35). It was no better than one-to-one human tutoring (-0.11) or small-group teaching (0.05).
Why it matters for us: Software can beat a lecture, but not a good tutor, so the coach should replace passive study time rather than teacher time.
Why it is not in the plan: background only; the plan cites VanLehn 2011 and Kulik and Fletcher 2012

Do Intelligent Tutoring Systems Benefit K-12 Students? What US school studies say about tutoring software.

Leite et al. 2025, arXiv preprint. Download · Source

Who was studied: 18 studies with 77 effect sizes covering 11 tutoring-software products in US elementary, middle and high schools.
How: Meta-analysis, plus a machine-learning method (MetaForest) to find which study features predict bigger effects.
What it found: The average effect was g = 0.27 (a quarter of a standard deviation), similar for low achievers, but lower in studies that included rural schools. Worked-out examples were among the strongest predictors of larger effects.
Why it matters for us: A realistic school-level effect for tutoring software is about a quarter SD, and showing full worked solutions matters.
Why it is not in the plan: background only; US population

Education Technology: An Evidence-Based Review. Which kinds of ed-tech work in randomised trials.

Escueta, Quan, Nickow and Oreopoulos 2017, NBER Working Paper 23744. Download · Source

Who was studied: Randomised trials of education technology, mostly from rich countries with some from developing countries; 29 trials of computer-assisted learning.
How: Review of RCTs in four groups: access to devices, computer-assisted learning (practice software), behavioural nudges, and online courses.
What it found: Giving students computers alone had mixed and often limited effects on learning. Of 29 computer-assisted learning trials, 20 found gains, 15 of those in maths. Fully online courses did slightly worse than face-to-face ones; blended courses did not.
Why it matters for us: Devices are not the intervention; structured maths practice with a teacher present is the category most likely to work.
Why it is not in the plan: background only; the plan cites the later Rodriguez-Segura 2021 review

Research on the Use of Khan Academy in Schools. How nine California sites actually used Khan Academy.

Murphy, Gallagher, Krumm, Mislevy and Hafter 2014, SRI Education report. Download

Who was studied: Students and teachers at nine California sites, mostly grades 5-10 maths, in school years 2011-12 and 2012-13.
How: Implementation study with surveys, interviews and usage logs, plus exploratory correlations between Khan Academy use and test scores.
What it found: More Khan Academy use went with better-than-predicted test scores, lower maths anxiety and more confidence, but the authors say these links cannot show cause. The heaviest use came where schools set goals with consequences, monitored closely and gave long blocks of time; use peaked at about 90 minutes a week at one site.
Why it matters for us: Usage depends on adults setting targets and watching progress, so the coach needs a teacher-owned routine, not just availability.
Why it is not in the plan: correlational only, not causal

The Role of Mastery Learning in an Intelligent Tutoring System. Does moving on before mastery hurt?

Sales and Pane 2017 (revised 2018), arXiv preprint. Download · Source

Who was studied: High-school Algebra I students in the US Cognitive Tutor effectiveness trial (year two of a 147-school randomised trial).
How: Statistical re-analysis of the trial’s log data, estimating whether the software’s effect differed for students who often got promoted without mastering a section.
What it found: The tutor may work better for students who were promoted more often without mastery, but the authors cannot say why. The paper also restates the trial’s results: -0.10 SD (standard deviations, the usual unit for comparing test-score gains) in year one and 0.22 SD in year two.
Why it matters for us: Strict mastery gates are not proven, and a first year with no gain is normal for a new tool.
Why it is not in the plan: background only; mastery result inconclusive

Mastery Learning in Practice. What students actually did inside a mastery-based tutor.

Israni, Sales and Pane 2018, arXiv preprint. Download · Source

Who was studied: 4,460 high-school students in 18 treatment schools of the US Cognitive Tutor Algebra I trial.
How: Mostly descriptive analysis of software logs, plus an observational estimate of what happened when teachers moved students on before mastery.
What it found: Students often moved on without mastering sections and worked out of order, more so in year two when the effect was larger. Median use fell from 33.5 hours in year one to 23.8 in year two. Teacher reassignment before mastery was linked to lower post-test scores, varying a lot by classroom.
Why it matters for us: Expect real use to fall short of plan and to bend the mastery rules, so log teacher overrides rather than forbid them.
Why it is not in the plan: background only; observational

Performance Comparison of an AI-based Adaptive Learning System in China. Squirrel AI’s own short trial.

Cui, Xue and Thai 2019, arXiv (authors employed by Squirrel AI). Download · Source

Who was studied: Middle-school students aged 13-15 in Chengdu, China, maths: 203 randomised, 163 finished (90 of 101 treatment, 73 of 102 control). A second study compared English platforms.
How: Short randomised experiment over three days of a national holiday, comparing the Squirrel AI system with lessons from expert teachers.
What it found: Squirrel AI students gained 9.38 points on a 100-point test against 1.81 for the teacher group (g = 0.68, an effect size in standard deviations).
Why it matters for us: Company-run, three-day studies with uneven dropout are not evidence a JEE coach can lean on.
Why it is not in the plan: company-authored, three days, uneven dropout; not credible

Assessing the Impact of Brilliant.org in Colombia. A small study of Brilliant for Grade 10 maths.

de la Puente and Perez 2023, Mathematics Teaching Research Journal 15(2). Download · Source

Who was studied: 60 Grade 10 students (about age 15) from four public schools in Barranquilla, Colombia, learning linear algebra and matrices.
How: Quasi-experiment (groups not randomly assigned): 33 students used Brilliant.org and 27 used videos and other resources; final grades compared. The abstract is written in the future tense, like a plan.
What it found: The Brilliant group’s mean final grade was 87.4 against 79.6 for the comparison group. The paper’s own statistics conflict, with one passage reporting no significant difference between groups.
Why it matters for us: It is the only academic study of Brilliant found, and it is too small and inconsistent to justify adopting Brilliant.
Why it is not in the plan: tiny non-random study with internally inconsistent results

Learning science

Improving Students’ Learning with Effective Learning Techniques. Which ten study habits actually work.

Dunlosky, Rawson, Marsh, Nathan and Willingham 2013, Psychological Science in the Public Interest 14(1). Download · Source

Who was studied: Hundreds of studies, mostly lab studies with undergraduates, some in schools, across ages and subjects.
How: Narrative review that rates 10 study techniques by how well and how widely they work.
What it found: Practice testing and spacing study out over time were rated ‘high utility’. Interleaving (mixing problem types), self-explanation and elaborative questioning were ‘moderate’. Rereading, highlighting and summarising were ‘low’.
Why it matters for us: The coach should steer study time to self-testing and spaced review, not rereading notes or solutions.
Why it is not in the plan: background only; the plan cites the meta-analyses directly

Test-Enhanced Learning. Testing yourself beats rereading for long-term memory.

Roediger and Karpicke 2006, Psychological Science 17(3). Download · Source

Who was studied: 120 US undergraduates aged 18-24 (Experiment 1), reading short prose passages.
How: Lab experiment: after a first reading, students either reread the passage or took a recall test without feedback, then were tested 5 minutes, 2 days or 1 week later.
What it found: After one week, tested students recalled 56% against 42% for rereaders (d = 0.83, an effect size in standard deviations). After 5 minutes, rereading looked better.
Why it matters for us: Retrieval practice will look worse on the day and better a week later, so the coach should not judge sessions by same-day accuracy.
Why it is not in the plan: duplicate of a cited finding (Adesope 2017 and Rowland 2014 meta-analyses)

The Value of Applied Research. Low-stakes quizzes in a real middle school.

Agarwal, Bain and Chamberlain 2012, Educational Psychology Review 24. Download · Source

Who was studied: Over 1,400 students in grades 6-8 at one US middle school (Illinois), social studies and science, nine teachers over five years.
How: Review of classroom experiments in which each student was quizzed on some material and not on other material, then tested later.
What it found: At the end of the semester, one to two months later, students scored 79% on quizzed material against 67% on material that had not been quizzed.
Why it matters for us: Frequent low-stakes quizzing works in real schools, not just labs, which supports quiz-first coach sessions.
Why it is not in the plan: duplicate of a cited finding

Spaced Retrieval Practice in Nine Introductory STEM Courses. Does spacing out quiz questions help in real college courses?

Bego, Lyle et al. 2024, International Journal of STEM Education 11 (excerpts only). Download · Source

Who was studied: 578 students in nine introductory US college STEM courses (science, technology, engineering and maths).
How: Classroom experiment within students: quiz items were either massed in one quiz or spaced over several quizzes, then tested later. Only verbatim excerpts were captured; the PDF could not be saved.
What it found: Spacing helped significantly in only two courses (calculus for engineers and chemistry for health professions); 6 of 9 courses had positive effects. The pooled effect was significant only when calculus was included, where feedback was richer and questions were fill-in-the-blank, not multiple choice.
Why it matters for us: Spacing is fragile in real STEM courses; the coach’s review questions should ask for recall and give informative feedback.
Why it is not in the plan: background only; only excerpts were read

Interleaved Practice Improves Mathematics Learning. Mixing problem types beats practising one type at a time.

Rohrer, Dedrick and Stershic 2015, Journal of Educational Psychology. Download · Source

Who was studied: 126 grade 7 maths students in the US, over three months.
How: Classroom experiment: the same practice problems were given either mixed (interleaved) or grouped by type (blocked), followed by a surprise test 1 or 30 days later.
What it found: Mixed practice produced higher scores: d = 0.42 after 1 day and d = 0.79 after 30 days (effect sizes in standard deviations).
Why it matters for us: JEE papers mix topics, so the coach should mix chapters in practice sets once each type has been learned.
Why it is not in the plan: duplicate of a cited finding (Rohrer et al. 2020 RCT)

The Power of Feedback Revisited. Which kinds of feedback help students most.

Wisniewski, Zierer and Hattie 2020, Frontiers in Psychology 10. Download · Source

Who was studied: 435 studies with over 61,000 students, many ages and subjects.
How: Meta-analysis of feedback studies, broken down by the type of feedback given.
What it found: Feedback had a medium average effect (d = 0.48, in standard deviations). Feedback that explains what went wrong, why and what to do next scored d = 0.99; simple right-or-wrong correction 0.46; praise or punishment 0.24.
Why it matters for us: The coach’s value is in explaining the error and the next step, not just marking answers.
Why it is not in the plan: background only; the ranking is more reliable than the exact numbers

Test-taking

Hit or Miss? Test Taking Behavior in Multiple Choice Exams. Who skips questions under negative marking.

Akyol, Key and Krishna 2016, NBER Working Paper 22401. Download · Source

Who was studied: About 10% of Turkish university entrance exam takers in 2002, social-science track (around 40,000 students); five options, minus a quarter mark per wrong answer.
How: Economic model of when students answer or skip, estimated on administrative exam data.
What it found: Women and high-ability students are more risk averse and skip more. Removing negative marking would raise women’s share of the top 5% from 39.9% to 40.2%.
Why it matters for us: Under JEE marking a guess after eliminating one option is worth taking, and some strong students will skip too much unless coached.
Why it is not in the plan: background only; the attempt rule lives in the literature review

Gender Differences in Willingness to Guess. Do women skip more questions when wrong answers cost marks?

Baldiga 2014, Management Science (working-paper copy). Download · Source

Who was studied: 406 adults in a Harvard Business School lab, US, answering SAT II subject-test practice questions.
How: Lab experiment varying whether wrong answers carried a small penalty and whether the test was framed as an SAT.
What it found: With no penalty everyone answered every question. With a small penalty, women answered significantly fewer questions than men, with no difference in knowledge or confidence; skippers scored lower given what they knew.
Why it matters for us: The coach should teach and track an explicit attempt rule, checking in Avanti mock data whether girls skip more.
Why it is not in the plan: background only; gender pattern contested by later work

Gender Differences in Strategic Behaviour under Competitive Pressure. Omissions in real Finnish entrance exams.

Pekkarinen 2014, IZA Discussion Paper 8018. Download · Source

Who was studied: 16,405 applicants (45% women) to economics and business programmes at nine Finnish universities, multiple-choice exams with minus marks.
How: Observational analysis of exam records, using a statistical model of each applicant’s chance of answering each item correctly.
What it found: Women omitted more items and were less likely to gain entry, given school marks. They strayed further than men from the number of answers that would maximise their chance of entry, because they answered too few.
Why it matters for us: Over-skipping shows up in a real high-stakes exam, so attempt coaching may matter for JEE scores.
Why it is not in the plan: background only; not from India

India and context

Top of the Class: The Importance of Ordinal Rank. Your rank in class shapes later results.

Murphy and Weinhardt 2013, CEP Discussion Paper 1241 (published Review of Economic Studies 2020). Download · Source

Who was studied: Over 2 million pupils in England tracked from primary to secondary school.
How: Observational study of administrative data, comparing pupils with the same test score but different rank within their primary school.
What it found: Moving from the 25th to the 75th percentile of rank in primary school, at the same test score, raised later secondary scores by 0.2 within-pupil standard deviations. Boys gained four times more from being top than girls; the mechanism appears to be self-confidence.
Why it matters for us: Rank within a class affects confidence and later results, which supports the plan’s choice of showing no leaderboards.
Why it is not in the plan: duplicate of a cited finding (Fang 2018, Azmat 2016)