Adaptive platforms and the pedagogy behind each tool
Adaptive practice software works when students actually practise on it, and the strongest predictor of whether they practise is not the software but an adult who owns a protected, supervised slot. The cleanest evidence for Avanti’s setting is two Indian RCTs: Mindspark in Delhi (0.36 SD in maths after 4.5 months, intent-to-treat, with an instructor in the room) and Khan Academy in residential government middle schools in Uttar Pradesh, where adding a dedicated lab-in-charge raised practice from 7.2 to 47.4 minutes a week and maths scores by about 0.44 SD (Muralidharan, Singh & Ganimian 2017; Oreopoulos, Keyes-Krysakowski & Agarwal 2026). Both trials are with Grade 6-9 students on remedial or grade-level maths, not with selected Grade 11-12 students on JEE content, and US trials on high-school algebra are smaller (about 0.2 SD, and only in the second year; Pane et al. 2014 as restated in Sales & Pane 2018). Brilliant, Alcumus and Alpha School have no independent evaluation; Alpha’s published numbers are its own MAP reports on students who mostly started at the 97th-99th percentile, and its model runs on paid adult guides, cash-like incentives and about 3.5 hours a day of supervised time. An AI tutor added on top of good practice software did not add learning beyond practice alone in the one large RCT, and students rarely talked to it when stuck (Oreopoulos & Low 2026).
Evidence table
Section titled “Evidence table”| Study | Design and population | n | Effect (unit) | What it means for the coach |
|---|---|---|---|---|
| Muralidharan, Singh & Ganimian 2017 (Mindspark, NBER w22923) | Lottery RCT; after-school centres in Delhi; Grades 6-9 from public schools; 45 min adaptive software + 45 min group instruction, 6 days a week | 619 | ITT 0.36 SD maths, 0.22 SD Hindi after 4.5 months; IV estimate 0.59 SD maths for 90 days attended; mean attendance 58% | Adaptive targeting works in India when an adult runs the room; math gains were on below-grade items, not grade-level exams |
| Oreopoulos, Keyes-Krysakowski & Agarwal 2026 (Khan Academy India, NBER w34683) | School-level RCT; residential government middle schools in Uttar Pradesh; all schools had Khan Academy, treatment got a lab-in-charge | 5,535 students, 74 schools | Usage 7.2 to 47.4 min/week; ITT 0.33 to 0.47 SD maths over 31 weeks | Closest analogue to JNV: the effect came from protected, supervised time and an owner, not from the software, which both arms had |
| Oreopoulos & Low 2026 (Khanmigo, NBER w35620) | Two-year cluster RCT; 18 Tennessee middle schools; Khan Academy with AI tutor in daily remedial maths periods | 6,902 student-terms | 0.06 to 0.08 SD per year; 0.14 SD implied for a full year of active use; similar to Khan without AI | Students messaged the AI tutor in only 17% of sessions where they made a mistake; access to an AI tutor is not use of it |
| Oreopoulos, Gibbs, Jensen & Price 2024 (NBER w32388) | Two field experiments; US elementary and middle school; weekly coaching for teachers on mastery learning with Khan Academy | ~11,000 students (per Khan’s summary) | ITT 0.12 to 0.22 SD maths | Gains concentrated in classrooms averaging 35+ min/week; teacher buy-in and a fixed practice slot predicted dosage |
| Grimaldi, Weatherholtz & Hill 2022 (Khan MAP Accelerator, EDM) | Observational, difference-in-differences; US Grades 3-8, 2020-21 | 181,359 | 0.26 SD for 30+ min/week vs <15 min/week | Only 5% of students reached the recommended dose; 41% never used it |
| Yamkovenko & Bergstrom 2024 (Khan Academy efficacy blog) | Company-reported observational study, US Grades 3-8, 2022-23 | ~350,000 | ~20% greater-than-expected MAP growth at 18+ hours/year | Only ~9% reached the dose; self-reported, not independent |
| Murphy et al. 2014 (SRI, Khan Academy in schools) | Implementation study, 9 California sites, correlational outcome analysis | 1,694 to 2,246 students per year | Positive associations only; authors say not causal | Highest use came with mandated goals, consequences, close teacher monitoring and 90-minute blocks |
| Roschelle, Feng, Murphy & Mason 2016 (ASSISTments) | Cluster RCT; 43 Maine schools; Grade 7 maths homework with feedback and teacher reports | 2,850 | 0.18 SD (Hedges g); 0.29 SD for students at or below median | Light-touch practice with immediate feedback plus a teacher report on common wrong answers is enough to move scores; actual use was 14 hours a year against 18-24 expected |
| Sales & Pane 2018, restating Pane et al. 2014 (Cognitive Tutor Algebra I) | Cluster RCT; 147 schools, 7 states; high and middle school algebra | ~18,700 high-school students | High school: -0.10 SD year 1, 0.22 SD year 2 | Expect a null first year; teachers need a year to learn the tool |
| Israni, Sales & Pane 2018 (CTAI log data) | Descriptive and observational analysis of the CTAI trial logs | 4,460 | Median use 33.5 h (year 1) and 23.8 h (year 2); reassignment before mastery associated with lower post-test | Students often moved on without mastery; dosage fell in year 2 |
| VanLehn 2011 | Review of experiments comparing tutoring types | Many small trials | vs no tutoring: human 0.79, step-based ITS 0.76, substep 0.40, answer-based 0.31 (d) | Feedback on each step of a solution is the active ingredient, not a right/wrong mark at the end |
| Kulik & Fletcher 2012 (IDA report D-4664, precursor of the 2016 RER meta-analysis) | Meta-analysis of 45 controlled ITS evaluations | 45 studies | Mean 0.60 SD; Cognitive Tutor 0.72 on local tests vs 0.10 on standardized tests | Gains on the system’s own tests overstate transfer; judge the coach on independent mocks |
| Ma, Adesope, Nesbit & Liu 2014 | Meta-analysis of ITS vs other instruction | 107 effect sizes, 14,321 learners | g = 0.41 overall; vs large-group 0.42; vs human tutoring -0.11; vs small-group 0.05 | ITS beats lecture, not a teacher with a small group |
| Leite et al. 2025 (arXiv) | Meta-analysis of US K-12 ITS studies | 18 studies, 77 effect sizes | g = 0.27; lower in rural schools | Realistic US K-12 size is about a quarter SD; worked examples a top moderator |
| Escueta, Quan, Nickow & Oreopoulos 2017 (NBER w23744) | Review of RCTs in ed-tech | 29 CAL RCTs | 20 of 29 positive; 15 of those in maths | CAL is the most promising ed-tech category, mainly for maths; access to devices alone does little; fully online slightly worse than face-to-face |
| Rodriguez-Segura 2021 draft (published WBRO 2022) | Review of causal ed-tech studies in developing countries | 81 core studies | Self-led learning median 0.29 SD | Dosage, adaptivity level and incentives are under-tested; one study found doubling dosage did not help |
| Alpha School 2025 MAP report (own data) | Self-generated NWEA MAP growth report, no comparison group | 148 math growth events, K-8 | Observed/projected growth 1.3x to 2.2x in maths by grade (my calculation from the table) | Most grades started at the 97th-99th percentile; no Grade 9-10 data; not an evaluation |
| ACX reader review 2025, Schoales 2026 (Fordham), Aldeman 2026 (The 74) | Parent account and two commentaries | n/a | n/a | Documented model: ~3.5 h supervised mornings, IXL for most work, 5:1 guides, “bucks” paid only at 80%+ accuracy; no outside study exists |
| Cui, Xue & Thai 2019 (Squirrel AI, company authors) | Short RCT during a vacation, China, Grade 8 maths | 203 randomised, 163 finished | Treatment > expert teacher | Three days, company-authored, 11% vs 28% attrition: not credible evidence |
| AoPS 2020s blog on Alcumus; Brilliant.org homepage; de la Puente & Perez 2023 | Product documentation; a study protocol with no results | n/a | None | No outcome evaluation exists for Alcumus or Brilliant |
| Gibney & Bojorquez 2022 (WestEd, Beast Academy) | AoPS-commissioned matched quasi-experiment, Grades 3-6 | one district | Positive, significant after two years | A different AoPS product for younger children; not evidence for Alcumus |
What the evidence says
Section titled “What the evidence says”The tools differ less in algorithm than in who makes students use them
Section titled “The tools differ less in algorithm than in who makes students use them”Mindspark, Khan Academy, ASSISTments, Cognitive Tutor, Alcumus and Squirrel AI share one learning model. Each holds a large bank of items tagged by skill, estimates what the student knows from their answers, picks the next item at the edge of that knowledge, gives immediate feedback, and brings back old skills for review. Mindspark adds error-pattern diagnosis from millions of answers and pitches content to the student’s actual level, which in Delhi was 2.5 grade levels below Grade 6 standard and 4.5 below by Grade 9 (Muralidharan, Singh & Ganimian 2017). ASSISTments “skill builders” require a streak of three correct answers and recheck retention at 1 and 2 weeks (Roschelle et al. 2016). Alcumus rates each student per topic on a 0-100 scale using an Elo-style model, mixes in review problems from passed topics, and AoPS itself says it “works best when wrapped in some amount of teaching” (AoPS blog). Alpha’s platform, by a parent’s account, is mostly IXL behind one login, with spaced review, an 80% mastery threshold and remote coaching calls; it uses no generative AI (ACX review 2025).
The outcome differences across these tools track implementation more than design. Kulik & Fletcher (2012) found that programmes “that paid careful attention to implementation were significantly more effective”. The Khan India trial is the sharpest test: every school had the same free software, and only the schools with a dedicated adult got the gains (Oreopoulos, Keyes-Krysakowski & Agarwal 2026). In the year before that trial, with training, WhatsApp support and an awards campaign but no dedicated person, only 44% of 27,309 registered students opened Khan Academy even once (same paper).
Most students stop using platforms unless someone protects the time
Section titled “Most students stop using platforms unless someone protects the time”Usage falls short of plan in every study that reports it. Mindspark lottery winners attended 58% of possible days and 13% attended one day or less (Muralidharan, Singh & Ganimian 2017). ASSISTments students averaged 14 hours a year against 18-24 hours planned (Roschelle et al. 2016). In Khan MAP Accelerator, 5% of 181,359 students reached 30 minutes a week and 41% never used it; usage was “mostly predicted by District/School/Teacher factors” (Grimaldi, Weatherholtz & Hill 2022). Khan’s own 2022-23 figure is 9% at the recommended dose (Yamkovenko & Bergstrom 2024). Cognitive Tutor median use fell from 33.5 hours in year 1 to 23.8 in year 2 (Israni, Sales & Pane 2018). In the SRI Khan study, the site with the highest use had one-to-one devices, mandated goals with consequences, close teacher monitoring and 90-minute maths blocks (Murphy et al. 2014).
Dose matters, but past a threshold the evidence is thin. Khan’s teacher-coaching RCTs found gains in classrooms averaging at least 35 minutes a week (Oreopoulos et al. 2024); the India RCT moved schools to 47 minutes (Oreopoulos et al. 2026). The Mindspark IV estimate implies larger gains with more days attended, under added assumptions (Muralidharan, Singh & Ganimian 2017). Rodriguez-Segura (2021 draft) reports that the one study testing a doubled dose, Bettinger et al. 2020, found no further gain. I have not opened Bettinger et al.
An AI tutor bolted onto practice software did not add learning
Section titled “An AI tutor bolted onto practice software did not add learning”In the Tennessee Khanmigo RCT, 96% of students tried the AI tutor, but the median student messaged it on a third of practice days and in 17% of sessions where they made a mistake; messages were mostly bare answers or clicks on suggested prompts. Effects (0.06-0.08 SD a year) resembled Khan without AI (Oreopoulos & Low 2026). The population is US remedial middle-school maths, not motivated JEE aspirants, so the size may differ; the pattern of low voluntary engagement is the finding to carry.
What the effect sizes are, and on what
Section titled “What the effect sizes are, and on what”Indian RCT effects are large because the counterfactual is weak and content is pitched at the student’s level. Mindspark’s maths gains appeared only on below-grade-level questions, with no gain on grade-level school exams in maths (Muralidharan, Singh & Ganimian 2017). The Khan India gains were “broad-based across achievement levels and question difficulty”, on an independent test, for Grades 6-8 (Oreopoulos et al. 2026). Oreopoulos et al. (2026) also report that Mindspark in Uttar Pradesh government schools reached 0.22 SD over 18 months, and that usage dropped about 50% when lab-in-charge presence was cut; I have not opened that Muralidharan & Singh (2025) paper.
On harder secondary content the effects are smaller and slower. Cognitive Tutor Algebra I, a full curriculum with 40% computer time, had no effect in year 1 and about 0.2 SD in year 2 (Pane et al. 2014, as restated by co-author in Sales & Pane 2018). Meta-analyses put ITS at g = 0.41 against all comparisons, 0.42 against lecture, and no better than human tutoring (-0.11) or small-group teaching (0.05) (Ma et al. 2014); the US K-12 average is g = 0.27, lower in rural schools (Leite et al. 2025). Kulik & Fletcher (2012) show the measurement trap: Cognitive Tutor averaged 0.72 SD on locally developed tests and 0.10 on standardized tests.
VanLehn (2011) explains why tool design still matters at the margin: tutoring that gives feedback on each step of a solution (d = 0.76 vs no tutoring) beats tutoring that only marks the final answer (0.31), and comes close to human tutoring (0.79). Most of the studies he pooled are short and many are with college students.
Mastery gating in practice
Section titled “Mastery gating in practice”Mastery gating is the shared design idea, but real use bends it. In the Cognitive Tutor trial students often moved on without demonstrating mastery and worked units out of order, more so in year 2 when the effect was larger; teacher reassignment before mastery was associated with lower post-test scores, an observational result that varied across classrooms (Israni, Sales & Pane 2018). A companion analysis found the tutor may be more effective for students who were promoted more often, without being able to say why (Sales & Pane 2017). Khan’s mastery levels (attempted, familiar, proficient, mastered) correlate with gains in its own data (Yamkovenko & Bergstrom 2024), but that is company-reported and observational.
Alpha School and “2 Hour Learning”: claimed vs documented
Section titled “Alpha School and “2 Hour Learning”: claimed vs documented”Claimed on its websites: students “learn 2x in 2hrs”, “rank in the top 1% nationwide”, and the top 20% show “6.5x growth” (Alpha and 2 Hour Learning homepages). Commentators report a “2.6 times” MAP growth claim (Aldeman 2026; Schoales 2026).
Documented: Alpha publishes one NWEA MAP Student Growth Summary for 2024-25, generated by Joe Liemandt from Alpha’s own testing. It covers 148 maths growth events from K to Grade 8 and none for Grades 9-10. Fall achievement percentiles in maths were already 97-99 in eight of nine grades (82 in Grade 5). Observed growth was 1.3x to 2.2x NWEA’s projected growth in maths and 1.1x to 1.9x in reading, depending on grade (my arithmetic from the published table; Alpha may compute its 2.6x figure another way). There is no comparison group and no outside study (Schoales 2026). The one columnist sympathetic to the model is a policy adviser to NWEA, which makes the MAP test, and discloses it (Aldeman 2026).
How the motivation actually works, from a parent who enrolled three children: mornings run 8:30 to about noon to fit in two hours of focused work in 20-30 minute bursts; most lessons are IXL; guides are paid USD 60,000-150,000 at about a 5:1 ratio; remote coaches take calls when the system detects repeated errors; students earn in-house currency only when they complete the lesson and score 80% or more, to stop rushing; and the home version, without guides or incentives, showed about 1x normal growth in Alpha’s own early data (ACX review 2025). That review is one anonymous parent’s account. Taken at face value, Alpha’s active ingredients are selection, adult supervision, incentives tied to accurate work, and spaced mastery practice, not the stitching of platforms.
Squirrel AI, Brilliant, Alcumus
Section titled “Squirrel AI, Brilliant, Alcumus”Squirrel AI’s open-access evidence is a company-authored paper: 203 Grade 8 students randomised for a three-day vacation study, with 90 of 101 treatment and 73 of 102 control students finishing (Cui, Xue & Thai 2019). An SRI-authored RCT in Interactive Learning Environments (Wang et al. 2020) reports gains over expert-teacher instruction; I could not open it, so its size and design are not stated here. I found no outcome evaluation of Brilliant; the only academic item is a protocol for a 60-student quasi-experiment in Colombia that reports no results (de la Puente & Perez 2023), and Brilliant’s site says “research-backed” without citing a study. I found no evaluation of Alcumus; the one AoPS-commissioned study is of Beast Academy, a different product for Grades 3-6 (Gibney & Bojorquez 2022).
Where the evidence is weak or contested
Section titled “Where the evidence is weak or contested”- No RCT tests adaptive software on JEE-level physics, chemistry or maths, or on Grade 11-12 students in India. Every Indian RCT here is Grades 6-9.
- The Khan India and Khanmigo trials are NBER working papers co-authored with Khan Academy staff or its research partners, not yet peer reviewed. The India trial randomised 74 schools; the abstract says 83 residential schools were in the programme.
- The components are rarely separated. Mindspark’s authors say the effect is of the blended package and that the adult “may be essential” for attendance and time on task (Muralidharan, Singh & Ganimian 2017). Rodriguez-Segura (2021 draft) reports that in one Chinese study the ed-tech arm was no better than extra paper practice; I have not opened that study.
- Effects on the platform’s own tests overstate transfer (Kulik & Fletcher 2012). Most Khan dosage evidence is observational and self-selected.
- Mastery gating’s own effect is unresolved. The CTAI logs point both ways (Israni, Sales & Pane 2018; Sales & Pane 2017).
- Incentive evidence for ed-tech is thin. Rodriguez-Segura (2021 draft) cites one study (Hirshleifer 2016, not opened) where rewarding inputs beat rewarding outputs. Alpha’s incentive system is described, not evaluated.
- Engagement decay over a full year is documented mostly as usage below plan, not as a time series for Indian adolescents.
Design rules for the Avanti coach
Section titled “Design rules for the Avanti coach”- Schedule the coach as a protected, supervised slot with a named adult owner, not as optional access. Same software, an owner, 6.6x the usage and about 0.44 SD (Oreopoulos et al. 2026); highest Khan use came with mandated goals and teacher monitoring (Murphy et al. 2014).
- Budget for realised use at about half of scheduled use, and track it weekly. Mindspark attendance was 58%; ASSISTments 14 of 18-24 planned hours; 5-9% of Khan users reached the recommended dose (Muralidharan et al. 2017; Roschelle et al. 2016; Grimaldi et al. 2022; Yamkovenko & Bergstrom 2024).
- Target at least 45 minutes a week of actual problem practice per student, measured as attempts, not logins. Gains appeared at 35-47 minutes a week (Oreopoulos et al. 2024; 2026).
- Make practice the default and the chat optional. Students rarely used the AI tutor when stuck, and the AI layer added nothing over practice alone (Oreopoulos & Low 2026).
- Give feedback at each step of a worked solution, not only right or wrong at the end. Step-based tutoring 0.76 vs answer-based 0.31 (VanLehn 2011); worked examples are a top moderator (Leite et al. 2025).
- Pitch each practice set at the student’s measured level, even below the syllabus point. Mindspark’s gains came from below-grade targeting (Muralidharan et al. 2017).
- Give teachers a weekly report of common wrong answers by item, and have them reteach those in class. That was the teacher half of the ASSISTments intervention, 0.18 SD (Roschelle et al. 2016).
- Use mastery thresholds, but let a teacher move a student on and log it. Students often advanced without mastery in practice, with mixed results (Israni, Sales & Pane 2018; Sales & Pane 2017).
- If effort is rewarded, pay only for work done at an accuracy floor. Alpha’s currency pays only at 80%+ to stop rushing (ACX review 2025); input rewards beat output rewards in one study (Hirshleifer 2016, via Rodriguez-Segura 2021 draft).
- Judge the pilot on independent mocks, not on the coach’s own quizzes. Local tests showed 0.72 SD where standardized tests showed 0.10 (Kulik & Fletcher 2012).
- Plan for a weak first cycle. Cognitive Tutor was null in year 1 and about 0.2 SD in year 2 (Pane et al. 2014 via Sales & Pane 2018); ASSISTments gave teachers a warm-up year (Roschelle et al. 2016).
- Set expectations from secondary-level evidence, not middle-school remediation. About 0.2 SD on harder content (Pane et al. 2014) and g = 0.27 for US K-12 ITS (Leite et al. 2025), against 0.36-0.47 SD in Indian Grade 6-9 remediation.
Where the current spec conflicts with the evidence
Section titled “Where the current spec conflicts with the evidence”- Section 3, “nudge, don’t control”, with students on regular Claude. The only large RCT of an AI tutor inside practice software found students rarely engaged it when stuck (Oreopoulos & Low 2026), and the Indian gains came from supervised, protected time (Oreopoulos et al. 2026). Nudges inside a chat do not substitute for a scheduled, supervised slot. The spec’s own risk 4 (“without a weekly teacher check, usage fades”) understates this: the evidence says a weekly check is not enough; someone has to run the sessions.
- Section 8b, the Alpha-inspired multi-platform model. Alpha’s results are self-reported on pre-selected high scorers; its documented ingredients are 5:1 paid guides, 3.5-hour supervised mornings, incentives gated on 80% accuracy and mostly a single practice tool (IXL), and its unsupervised home version showed about 1x growth (Alpha 2025 MAP report; ACX review 2025). Brilliant and Alcumus have no outcome evaluation. A Chrome-plugin tracker across platforms adds the engineering cost and the fragmentation without the ingredients that plausibly drive Alpha’s numbers. The spec lists motivation as design question 4; the evidence says it is the main question.
- Section 8c, the 0.15-0.37 SD expectation inherited from the Learning Framework. That range comes from Mindspark, which was Grades 6-9 in Delhi with gains only on below-grade maths items (Muralidharan, Singh & Ganimian 2017). It is not secondary-level JEE evidence. The Big Bet document also cites the Khan India RCT as evidence for “Indian secondary students” and for “mastery-gating philosophy”; the trial is Grades 6-8 and tests implementation support, since both arms had the same mastery-based software (Oreopoulos et al. 2026). The Big Bet document’s statement that Muralidharan et al. found “teacher-intensive differentiation is the weakest point” does not appear in the 2017 NBER paper I opened.
- Section 10, success metrics “plan completion” and “accuracy change on target chapters”. If accuracy is measured on the coach’s own quizzes, it will overstate learning (Kulik & Fletcher 2012). Primary outcome should be independent AIAT or mock scores.
- Section 2, 2-3 hours device time per week. This is above the 35-47 minute thresholds where gains appear (Oreopoulos et al. 2024; 2026), which is good, but realised use is likely to be about half (rule 2). The spec should say who protects the slot at the centre and what happens when the slot is lost to tests or events. The handoff concept already names the real barrier: teachers release tablets only for tests.
- Section 4, problem 2, practice “in Claude”. Free-text chat practice is closer to answer-based tutoring than to step-based tutoring unless each problem is broken into steps with feedback (VanLehn 2011). The spec should specify step-level checking as well as a quiz score.
- The handoff concept’s 10-week pilot to JEE Main. Cognitive Tutor showed nothing in year 1 (Pane et al. 2014 via Sales & Pane 2018). A one-term pilot can test usage and level movement; it is unlikely to detect a JEE effect. Its effort rewards (streaks, points) are consistent with rule 9 only if tied to an accuracy floor.
References
Section titled “References”- Aldeman, C. (2026, January 8). What public schools and parents can learn from a $40,000-a-year private school. The 74. https://www.the74million.org/article/what-public-schools-and-parents-can-learn-from-a-40000-a-year-private-school/ . Opened 2026-10-26. Archived: T74_alpha.html
- Alpha School (2025). Student Growth Summary Report, Aggregate by District, Spring 2024-2025 (NWEA MAP Growth, generated 15 June 2025). https://go.alpha.school/hubfs/MAP%20Results%20-%2024%2025/2025%20NWEA%20MAP%20results.pdf . Opened 2026-10-26. Archived: ALPHA2025_map_results.pdf
- Alpha School homepage, https://alpha.school/ ; 2 Hour Learning homepage, https://2hourlearning.com/ ; 2 Hour Learning results page, https://2hourlearning.com/results/ . Opened 2026-10-26. Archived: ALPHA_site_home.html, ALPHA_2hourlearning_home.html, ALPHA_2hl_results.html
- Anonymous ACX reader (2025, June 27). Your Review: Alpha School. Astral Codex Ten. https://www.astralcodexten.com/p/your-review-alpha-school . Opened 2026-10-26. Archived: ACX2025_alpha_parent_review.html
- Art of Problem Solving. Alcumus: a peek under the hood of our adaptive learning tool (blog). https://artofproblemsolving.com/blog/articles/alcumus-a-peek-under-the-hood-of-our-adaptive-learning-tool . Opened 2026-10-26. Archived: AOPS_alcumus_underhood.html
- Brilliant.org homepage. https://brilliant.org/ . Opened 2026-10-26. Archived: BRIL_site_home.html
- Cui, W., Xue, Z., & Thai, K.-P. (2019). Performance comparison of an AI-based adaptive learning system in China. arXiv:1901.10268. https://arxiv.org/abs/1901.10268 . Opened 2026-10-26. Archived: SQAI2019_arxiv_company.pdf
- de la Puente, M., & Perez, H. (2023). Assessing the impact of Brilliant.org on enhancing mathematics academic performance among high school students in Colombia: a quasi-experimental study. Mathematics Teaching Research Journal, 15(2). https://files.eric.ed.gov/fulltext/EJ1394390.pdf . Opened 2026-10-26. Archived: BRIL_eric_protocol.pdf
- Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017). Education technology: an evidence-based review. NBER Working Paper 23744. https://www.nber.org/papers/w23744 . Opened 2026-10-26 (copy from gs://avantifellows-research-sources/nvs-lakshya/). Archived: EQNO2017_edtech_nber.pdf
- Gibney, T. T., & Bojorquez, J. C. (2022). Evaluation of Beast Academy in Mankato Area Public Schools. WestEd technical report for Art of Problem Solving. https://beastacademy.com/u/pdfs/marketing/WestEd%20Beast%20Academy%20Evaluation%20Technical%20Report%20Final.pdf . Opened 2026-10-26. Archived: WESTED2022_beastacademy.pdf
- Grimaldi, P., Weatherholtz, K., & Hill, K. M. (2022). Estimating the causal effects of Khan Academy MAP Accelerator across demographic subgroups. Proceedings of the 15th International Conference on Educational Data Mining, 839-846. https://doi.org/10.5281/zenodo.6852932 . Opened 2026-10-26. Archived: KHAN2022_edm_mapaccel.pdf
- Israni, A., Sales, A. C., & Pane, J. F. (2018). Mastery learning in practice: a (mostly) descriptive analysis of log data from the Cognitive Tutor Algebra I effectiveness trial. arXiv:1802.08616. https://arxiv.org/abs/1802.08616 . Opened 2026-10-26. Archived: SP2018_mastery_logdata_arxiv.pdf
- Khan Academy (n.d.). Multiple studies show Khan Academy drives learning gains (blog). https://blog.khanacademy.org/multiple-studies-show-khan-academy-drives-learning-gains-evidence-for-our-platforms-effectiveness/ . Opened 2026-10-26; used only to locate primary papers. Archived: KHAN_multiple_studies_blog.html
- Kulik, J. A., & Fletcher, J. D. (2012). Effectiveness of intelligent tutoring systems. IDA Document D-4664, Institute for Defense Analyses. https://www.ida.org/-/media/feature/publications/e/ef/effectiveness-of-intelligent-tutoring-systems/d-4664.ashx . Opened 2026-10-26. Archived: KF2016_its_meta.pdf. (Precursor report of Kulik & Fletcher 2016, Review of Educational Research 86(1), which I did not open.)
- Leite, W. L., Zhang, H., Rana, S., Hao, Y., Hatch, A. D., Kong, L., & Kuang, H. (2025). Do intelligent tutoring systems benefit K-12 students? A meta-analysis and evaluation of heterogeneity of treatment effects in the U.S. arXiv:2511.04997. https://arxiv.org/abs/2511.04997 . Opened 2026-10-26. Archived: ITS2025_k12_meta_arxiv.pdf
- Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: a meta-analysis. Journal of Educational Psychology, 106 (archived online-first copy marked Vol. 106, No. 3). https://doi.org/10.1037/a0037123 (copy at https://www.cs.uky.edu/~sgware/reading/papers/ma2014intelligent.pdf). Opened 2026-10-26. Archived: MANL2014_its_meta.pdf
- Muralidharan, K., Singh, A., & Ganimian, A. J. (2017). Disrupting education? Experimental evidence on technology-aided instruction in India. NBER Working Paper 22923 (revised July 2017). https://www.nber.org/papers/w22923 . Opened 2026-10-26 (copy from gs://avantifellows-research-sources/nvs-lakshya/). Archived: MSG2019_mindspark.pdf. (Published in the American Economic Review in 2019; the Learning Framework quotes 0.37 and 0.23 SD from that version, which I did not open.)
- Murphy, R., Gallagher, L., Krumm, A., Mislevy, J., & Hafter, A. (2014). Research on the use of Khan Academy in schools. SRI Education. Opened 2026-10-26 (copy from gs://avantifellows-research-sources/nvs-lakshya/). Archived: SRI2014_khan.pdf
- Oreopoulos, P., Gibbs, C., Jensen, M., & Price, J. (2024). Teaching teachers to use computer assisted learning effectively: experimental and quasi-experimental evidence. NBER Working Paper 32388. https://www.nber.org/papers/w32388 . Opened 2026-10-26. Archived: NBER_w32388.pdf
- Oreopoulos, P., Keyes-Krysakowski, O., & Agarwal, D. (2026). How in-school supervised ed-tech support produces massive learning gains: a Khan Academy field experiment in India. NBER Working Paper 34683. https://www.nber.org/papers/w34683 . Opened 2026-10-26. Archived: NBER_w34683.pdf
- Oreopoulos, P., & Low, N. (2026). One click away: AI tutoring with Khanmigo in a two-year school experiment. NBER Working Paper 35620. https://www.nber.org/papers/w35620 . Opened 2026-10-26. Archived: NBER_w35620.pdf
- Rodriguez-Segura, D. (2021, June draft). Educational technology in developing countries: a review of the evidence. Published as World Bank Research Observer 37(2), 2022, https://doi.org/10.1093/wbro/lkab011 . Draft at https://www.ictworks.org/wp-content/uploads/2023/03/Edutech-developing-countries.pdf . Opened 2026-10-26 (draft only). Archived: RS2022_ictworks.pdf
- Roschelle, J., Feng, M., Murphy, R. F., & Mason, C. A. (2016). Online mathematics homework increases student achievement. AERA Open, 2(4). https://doi.org/10.1177/2332858416673968 (ERIC EJ1194398). Opened 2026-10-26. Archived: RFMM2016_assistments.pdf
- Sales, A. C., & Pane, J. F. (2017, rev. 2018). The role of mastery learning in an intelligent tutoring system: principal stratification on a latent variable. arXiv:1707.09308. https://arxiv.org/abs/1707.09308 . Opened 2026-10-26. Archived: SP2017_mastery_cta1_arxiv.pdf. (Source of the restated Pane et al. 2014 effects: high school -0.10 and 0.22 SD, middle school -0.03 and 0.19 SD.)
- Schoales, V. (2026, September 11). Alpha School’s Theranos problem. Thomas B. Fordham Institute, Flypaper. https://fordhaminstitute.org/national/commentary/alpha-schools-theranos-problem . Opened 2026-10-26. Archived: FORD2025_alpha_theranos.html
- SRI International. Publication page for Wang et al. (2020). https://sri.com/publication/education-learning-pubs/when-adaptive-learning-is-effective-learning-comparison-of-an-adaptive-learning-system-to-teacher-led-instruction . Opened 2026-10-26 (abstract only; not used for any figure). Archived: SRI_squirrel_page.html
- VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197-221. https://doi.org/10.1080/00461520.2011.611369 (author copy via Wayback Machine of http://www.public.asu.edu/~kvanlehn/Stringent/PDF/EffectivenessOfTutoring_Vanlehn.pdf). Opened 2026-10-26. Archived: V2011_tutoring.pdf
- Yamkovenko, B., & Bergstrom, A. (2024, November). Khan Academy efficacy results, November 2024. Khan Academy Blog. https://blog.khanacademy.org/?p=18194 . Opened 2026-10-26. Archived: KHAN2024_efficacy_blog.html
All archived files are in the session scratchpad at sources/adaptive_platforms/, each with a .txt extraction beside it.
Leads, not opened
Section titled “Leads, not opened”- Muralidharan, K., Singh, A., & Ganimian, A. J. (2019). American Economic Review, 2019. Published version of the Mindspark RCT (reported 0.37/0.23 SD). Paywalled; the NBER working paper was used instead.
- Muralidharan, K., & Singh, A. (2025). Mindspark in Uttar Pradesh government schools. Known only through Oreopoulos et al. (2026): 0.22 SD over 18 months, usage down about 50% when lab-in-charges were cut. Not located.
- Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at scale. EEPA 36(2), 127-144. ERIC has no full text; RAND blocked automated download of WR-984. Results taken from Sales & Pane (co-authored by Pane).
- Kulik, J. A., & Fletcher, J. D. (2016). RER 86(1), 42-78. Paywalled; the 2012 IDA report was used.
- Escueta, M., Nickow, A., Oreopoulos, P., & Quan, V. (2020). Upgrading education with technology. JEL 58(4), 897-996. The AEA PDF route returned an HTML page; the 2017 NBER version was used.
- Wang, S., Christensen, C., Cui, W., Tong, R., Yarnall, L., Shear, L., & Feng, M. (2020). When adaptive learning is effective learning (Squirrel AI RCT). Interactive Learning Environments. Paywalled.
- Reich, J., & Ruipérez-Valiente, J. A. (2019). The MOOC pivot. Science 363(6423). MIT DSpace accepted manuscript returned server errors.
- Studies reported only through Rodriguez-Segura (2021 draft): Hirshleifer (2016) on input vs output rewards; Bettinger et al. (2020) on doubled dosage; Ma et al. (2020) on ed-tech vs paper practice in China; Linden (2008) on CAL as substitute vs complement in Gujarat.
- Eames, Brunskill, Yamkovenko, Weatherholtz & Oreopoulos (2026), PNAS, Khan Academy usage quasi-experiment; Ferman, Finamor & Lima (2019), Brazil; Büchel et al. (2019), El Salvador; Snipes et al. (2015). Listed on Khan’s blog; not opened.
- Any outcome evaluation of Alcumus or Brilliant.org: none found in ERIC, web search or the companies’ sites.
- The sci-hub fallback was refused by this session’s permission controls, so paywalled DOIs above remain leads.