Skip to content
Avanti Docs

Strand A: live vs video vs self-study vs blended/active (high-income, higher-ed, secondary STEM)

Searched 2026-11-01. Every number below was read in a copy archived in strandA/ (PDF verified %PDF; PMC and IRRODL full texts as JATS XML, HTML or EPUB; Noetel as the authors’ accepted-manuscript docx from OSF). Effect-size rows are in strandA.tsv (43 rows).

1. Synthesis: what we know, and how sure

Section titled “1. Synthesis: what we know, and how sure”

  1. The medium on its own does little when content and interactivity are held equal. Bernard et al. 2004 pooled 318 achievement outcomes (N = 54,775) comparing distance education with classroom teaching and found g+ = 0.01 [-0.007, 0.033]. Live remote teaching came out slightly worse (synchronous g+ = -0.10 [-0.15, -0.06]) and self-paced recorded work slightly better (asynchronous g+ = +0.05 [0.03, 0.08]). The spread was very wide, and most of it was explained by methodology and pedagogy rather than medium. In the only RCT-only review, Noetel et al. 2021 (83 studies), video that replaced a teacher gave g = 0.18 [0.02, 0.34], and g = 0.18 [0.01, 0.35] on knowledge tests. Small RCTs on single lectures (Schreiber 2010, Solomon 2004, Tolonen 2023) find a recorded lecture roughly equal to the same lecture given live. Confidence: moderate. Most tests are immediate and written by the researchers, and Noetel rated no study as low risk of bias.

  2. Video does clearly beat text alone. In Noetel, video that replaced static media such as text gave g = 0.51 [0.27, 0.75] (k = 27). Plain self-study from text is therefore the weakest of the three media. The same review reports skills (g = 0.44) gaining more than knowledge (g = 0.18). Confidence: low to moderate. These are short, lab-like studies, mostly in health sciences.

  3. Full-course field RCTs that replace live teaching show small losses, concentrated in weaker students. This is the result that matters most for Avanti.

    • Figlio et al.: live minus video of the same lectures = 1.4 points out of 100 (SE 1.2, n.s.), but 4.1 points (SE 1.5) for low achievers and 11.3 for Hispanic students.
    • Kofoed et al.: a live Zoom class versus in person lowered final grades by 0.215 SD (SE 0.084), and by 0.27 SD in the bottom quartile.
    • Cacault et al. (Switzerland): access to live-streamed lectures cut low-ability students’ share of exam questions correct by 1.9 points (ITT; ATT -0.18) and raised high-ability students’ by 2.5.
    • Le 2022 (Vietnam): pre-recorded lectures versus live online lectures gave -1.6 points for students below median GPA and about zero for those above.
    • Alpert et al., which I read only second-hand through Escueta et al.: purely online was 4-5 points below face-to-face.

    Confidence: moderate. The direction is consistent across 5 RCTs, all in intro economics or similar courses. The mechanism these papers point to is attendance, concentration and structure rather than how information is carried.

  4. Supplementing beats replacing, and blended designs roughly match face-to-face.

    • Noetel: video added on top of existing teaching gave g = 0.88 [0.62, 1.13]. The abstract of the same manuscript says 0.80, and the treatment groups usually got more total content.
    • Means et al.: blended against face-to-face gave +0.35 (k = 23), and purely online +0.14 (k = 28). These are confounded by extra time on task (+0.46 where online students spent more time).
    • In RCTs, Bowen et al.’s interactive online statistics course plus one hour a week face-to-face matched traditional teaching (CAOS +1 point, n.s., N = 605). Joyce et al. halved class time and lost about 0.2 SD (-2.5 pp; lower CI -4.2 pp).
    • Escueta et al.’s review of the experimental literature: fully online gives small losses, and blended has not been found to underperform.

    Confidence: moderate.

  5. Interactivity matters more than medium.

    • Noetel: video’s advantage disappears when the comparison class is more interactive (g = -0.07 [-0.38, 0.25]) and grows when the video comes with interaction (0.62 [0.13, 1.11]).
    • Active learning against lecture: g = 0.47 SD across 158 studies (Freeman 2014), 0.51 [0.32, 0.71] in randomized or crossover studies, and lecturing carries 1.95 times the odds of failing.
    • Deslauriers 2019 (RCT, physics): active learning +0.46 SD on an independently written test, even though students felt they had learned 0.56 SD less.
    • Theobald 2020: active learning shrinks the exam gap for underrepresented and low-income students from -0.62 SD to -0.42 SD (a 33% cut), but only at high intensity.

    Confidence: moderate to high on direction. Freeman’s trim-and-fill and fail-safe N held up. Size is less certain, because concept-inventory effects run larger than course-exam effects.

  6. Flipped classrooms help a little, and less after correcting for bias.

    • Låg and Sæle: g = 0.35 [0.31, 0.40] (k = 272), falling to 0.24 in adequately powered studies and 0.17 [0.12, 0.22] after trim-and-fill on those. That leaves about 0.2 SD.
    • van Alten: g = 0.36 [0.28, 0.44] (k = 114, I² = 88%, 76 of 114 studies not randomized). Effects were lower when flipping cut class time (-0.26) and higher when quizzes were added (+0.19).
    • Secondary-school subsets: 0.45 [0.27, 0.64] (k = 20, Låg); van Alten’s secondary-vs-higher-ed difference was +0.18, n.s. (k = 11).

  7. Self-paced interactive or AI tutoring: the strongest effects are also the least tested.

    • Kestin 2025: an at-home AI tutor beat an in-class active-learning lesson by 0.63 SD (0.73-1.3 after a quantile correction for ceiling). The test was immediate, researcher-designed, on 2 lessons, with 194 Harvard students.
    • Bastani 2025 (Turkish high school, opened in passing): unguarded GPT-4 practice lowered later unaided exam grades by 17%, while a tutor giving hints rather than answers largely avoided the harm.
    • Cognitive Tutor Algebra (blended): +0.21 SD in the second year of use, nothing in the first.
    • The lesson for an AI coach is that pedagogy (withholding answers, retrieval, structure) decides the outcome, not the medium.

  8. US secondary-school STEM RCTs give null-to-negative results for replacing teachers with online delivery.

    • Heppen 2017 (Chicago, students who had failed algebra): online credit recovery scored -0.19 SD on the proximal posttest and had lower credit recovery (66% vs 78%). On the delayed standardized PLAN test it was +0.08, n.s.
    • Kentucky’s hybrid Algebra I (cluster RCT, 47 schools): -0.09 SD on PLAN, n.s.

  9. The gaps across this literature are large. Few studies measure outcomes after a delay or on a standardised, high-stakes exam. Noetel found outcome timing did not moderate effects, Heppen’s gap shrank to zero on the delayed standardized test, and Tolonen found no differences after 5-7 weeks. No study tests JEE/NEET-like preparation. Several syntheses carry design problems:

    • Means: time-on-task confounding.
    • Gegenfurtner 2026: implausibly narrow confidence intervals and N reported two ways (3,823 and 3,283).
    • Noetel: publication bias was detected.
    • Online-course studies: students choose their mode unless randomized.

2. Opened sources

Section titled “2. Opened sources”

Column key: ES = effect size; “read at” = the page, table or section where I read the number.

Source (file) Design / population / country Comparison Outcome, delay Key ES [95% CI] (k / N) Heterogeneity / bias Key moderators Read at; short quote
Noetel et al. 2021, RER 91(2), doi:10.3102/0034654321990713 (noetel_2021_video_preprint.docx, accepted MS from OSF kynez) MA of RCTs, higher ed, multi-country Video replacing vs supplementing other teaching Knowledge/skill tests, mixed timing Replace g = 0.28 [0.14, 0.42] (83 studies, 166 effects); supplement 0.88 [0.62, 1.13] (34) 3PSM detects bias; nullifying would need 10.48x publication preference; no study low risk Replacing teacher 0.18; replacing text 0.51; control more interactive -0.07; skills 0.44 vs knowledge 0.18; timing n.s. Results + Table 1: “no benefit to video when the control condition was afforded more interactivity”
Means et al. 2010, US ED (means_2010_online.pdf, ERIC ED505824) MA, 51 contrasts (7 K-12), US Online/blended vs face-to-face Course learning All +0.24; pure online +0.14 (28); blended +0.35 (23) Heterogeneous; time on task confounds (+0.46 vs +0.19) Blended > pure online (Q = 4.98) pp. xiv, 18-19: “conditions generally differed on multiple dimensions”
Bernard et al. 2004, RER 74(3) (bernard_2004_distance.pdf, MS 27/10/04, Rochester copy) MA (fixed), 1985-2002, K-12 to adult Distance ed vs classroom Achievement 0.013 [-0.007, 0.033] (k = 318, N = 54,775); sync -0.10; async +0.05 Q = 1191, very wide spread (-1.31 to +1.41) Methodology and pedagogy matter more than media Tables 7-8: “effect sizes of essentially zero on all three measures”
van Alten et al. 2019, ERR 28 (vanalten_2019_flipped.pdf) MA, 114 studies (11 secondary), 76 not randomized Flipped vs non-flipped Assessed learning 0.36 [0.28, 0.44] (N = 20,318) I² = 88%; no strong bias signal Reduced class time -0.26; quizzes +0.19 Tables 3-4
Låg & Sæle 2019, AERA Open (lag_2019_flipped.pdf, ERIC) MA, 272 studies (20 secondary) Flipped vs lecture-based Learning; pass rate 0.35 [0.31, 0.40]; powered 0.24; trim-and-fill 0.17 [0.12, 0.22]; pass OR 1.55 I² = 83% Secondary 0.45; STEM 0.32; tested preparation 0.40 vs 0.31 Abstract, Table 3: “true weighted mean effect is somewhere around one fifth”
Freeman et al. 2014, PNAS (freeman_2014_active.html, PMC) MA, undergrad STEM (158 exam + 67 failure-rate studies) Active learning vs lecture Exams/concept inventories; failure 0.47 SD (158); randomized 0.514 [0.32, 0.71] (16); fail OR 1.95 (67) Trim-and-fill and fail-safe N robust Larger in small classes; concept inventories > exams Abstract, Table 1
Theobald et al. 2020, PNAS (theobald_2020_active.xml) Student-level MA, same instructor, US STEM Gap for underrepresented/low-income under lecture vs active Exams; passing Gap -0.62 [-0.69, -0.55] vs -0.42 [-0.48, -0.35] (15 studies, 9,238) Bayesian Only high-intensity active learning narrows the gap Results: “33% reduction in achievement gaps”
Deslauriers et al. 2019, PNAS (deslauriers_2019_feeling.xml) Crossover RCT, Harvard physics, N = 149 Active vs passive lecture Independent MCQ, immediate +0.46 SD — Feeling of learning -0.56 SD Table 3
Kestin et al. 2025, Sci Rep 15:17458 (kestin_2025_aitutor.xml) Crossover RCT, Harvard physics, N = 194 AI tutor at home vs in-class active learning Post-test, immediate 0.63 (linear); 0.73-1.3 (quantile) Ceiling effect Median 49 min vs 60 min class Results text
Figlio, Rush & Yin 2010, NBER w16089 (figlio_2013_online.pdf) RCT, volunteers, US microeconomics Live vs internet video of the same lectures Exams, end of course +1.44 pts live (SE 1.21), N = 296 — Low achievers 4.05**; Hispanic 11.28***; male 3.48** Tables 3-4. Escueta cites larger (~3/2.5 pt) gaps, likely from the published version
Joyce et al. 2014, NBER w20006 (joyce_2015_classtime.pdf) RCT, N = 725, Baruch Hybrid (half the class time) vs traditional Midterm + final -2.5 pp (~ -0.2 SD); lower bound -4.2 — Non-experimental attendance effect 2.5x inflated p. 17
Bowen et al. 2012, Ithaka S+R (bowen_2012_ilo.pdf) RCT, 6 campuses, N = 605 CMU interactive online + 1 hr/wk F2F vs traditional CAOS (standardized), final, pass +1, +2, +3 pp, all n.s. — “same basic results hold for subgroups” Figure 1 p. 18
Cacault et al. 2019, IZA DP 12298 (cacault_2021_distance.pdf) RCT by student x week, Geneva Access to live-streaming vs in class Exam items mapped to week ITT 0.001 (0.005) — Low ability -0.019 (0.009), ATT -0.18; high +0.025, ATT +0.25 Table 7
Kofoed et al. 2021, IZA DP 14356 (kofoed_2024_zooming.pdf) RCT, West Point, N = 551 Live Zoom vs in person Final grade (SD) -0.215 (0.084) — Final exam -0.147; bottom quartile -0.267 Tables 3-4
Heppen et al. 2017, JREE 10(2) (heppen_2017_creditrecovery.pdf) RCT, 1,224 Chicago 9th graders who failed algebra Online vs F2F credit recovery Posttest (end of course); PLAN (delayed, standardized) -0.19 posttest; +0.08 PLAN (n.s.) — Credit recovery 66% vs 78%; lower clarity Table 7
Cavalluzzo et al. 2012, NCEE 2012-4020 (cavalluzzo_2012_kyhybrid.pdf) Cluster RCT, 47 KY schools, 6,908 grade-9 students Hybrid online Algebra I vs usual PLAN (next fall) -0.09 (text) / -0.15 (table), p = .16 — Text and table disagree p. 67; Table 5.2
Pane et al. 2014, RAND WR-1050 addendum (pane_2013_cogtutor_wp.pdf) Cluster RCT, US high-school Algebra I Blended Cognitive Tutor vs usual Algebra posttest Cohort 2 +0.21 (0.10); cohort 1 -0.07 — Effect in year 2 only Table 3
Martin et al. 2021, IRRODL 22(3) (martin_2021_synchronous.epub) MA, 19 publications Synchronous online vs async; vs F2F Cognitive vs async 0.37 [0.06, 0.68] (k = 7); vs F2F -0.20 [-0.75, 0.35] (k = 4) Q significant; fail-safe N only — Table 5
Ebner & Gegenfurtner 2019, Front Educ (ebner_2019_webinar.pdf) MA, 5 studies Webinar vs F2F; vs async Knowledge 0.06 [-0.37, 0.49]; vs async 0.29 [0.05, 0.53] Tiny k — Results
Gegenfurtner et al. 2026, LID 126 (gegenfurtner_opus127485.pdf, CC BY) MA of 31 RCTs, mean age ~35 Webinar vs F2F; async; waitlist Cognitive vs F2F gc 0.19 (k = 12); vs async 0.25; knowledge vs F2F 0.47; skills vs F2F -0.21 Egger n.s.; CIs implausibly narrow Satisfaction 0.07; motivation -0.38 Tables 4, 6
Le 2022, Sustainability 14:2910 (le_2022_prerecorded.pdf, MPRA) RCT within/between students, Vietnam, N = 154 Pre-recorded vs live online Exam items -0.010 (0.003) — Low ability -0.016; high -0.003 Tables 3-4
Tolonen et al. 2023, BMC Med Educ 23:778 (finland_2023_remote.xml) 4-arm RCT, N = 301, Finland Vodcast vs live vs podcast vs live-stream MCQ, immediate and 5-7 wk Vodcast 22.5 (5.3) vs live 22.9 (6.1); podcast lower; delayed n.s. — — Abstract; Table 5
Schreiber et al. 2010, BMC Med Educ 10:68 (schreiber_2010_podcast.xml) Crossover RCT, 66 of 100 analysed, UK Podcast vs live lecture MCQ, immediate 88.8 vs 90.6; 86.7 vs 89.9 (n.s.) Ceiling effect Students preferred live Table 1
Solomon et al. 2004, BMC Med Educ 4:27 (solomon_2004_digitallecture.xml) RCT, N = 29 CD-ROM vs live lectures Exam n.s. Underpowered — Table 1
Bastani et al. 2025, PNAS 122 (bastani_2025_genai.xml) — sibling-scope lead RCT, ~1,000 grades 9-11 students, Turkey GPT-4 practice (Base / Tutor) vs none Unaided exam Base -17% — Tutor with hints mitigates Abstract
Escueta et al. NBER w23744 (escueta_2020_edtech.pdf) Review of experiments Online/blended vs F2F — Alpert 2016 (second-hand): online 4-5 pts below; blended no difference — — pp. 71-72, Table 4
Nordmann et al. 2019, Higher Educ (nordmann_2019_lecturecapture.pdf) Observational, UK Attendance and recording use Grades No ES; first-years benefit from both Correlational Weaker students helped by supplementary use Abstract
Alarifi & Song 2024, HSSC 11 (hssc_2024_online_inperson.pdf) Cohort comparison, 2020 vs 2021, Saudi Arabia Online vs in person Final tests Mixed by course Confounded by COVID and different exams — Abstract. Weak; excluded from TSV

3. Leads, not opened

Section titled “3. Leads, not opened”
Lead Why What I know (unverified)
Bettinger, Fox, Loeb & Taylor 2017, AER 107(9) “Virtual Classrooms” Large IV study of DeVry online vs in-person courses Every copy returned 403 (AEA, Harvard Scholar, CEPA Wayback). A search-engine summary said online lowers grades by about 1/3 SD and later grades too. Not verified.
Alpert, Couch & Harmon 2016, AER P&P 106(5) Three-arm RCT: online / blended / F2F Paywalled. Numbers above are only Escueta’s second-hand account.
Strelan, Osborn & Palmer 2020, ERR 30:100314 (flipped, 198 studies) Largest flipped meta-analysis Closed access. Search snippet: discipline effects range from IT g = 0.30 to humanities 0.98. Not verified.
Bernard et al. 2009, RER 79(3) — interaction treatments in distance education Mechanism (interaction) Closed.
Means et al. 2013, Teachers College Record (journal version of the US ED meta-analysis) Updated estimates Closed.
Zhang, Li, Pi & Yang 2025, ERR 48:100708, “Active learning strategies in video learning: a meta-analysis” Directly relevant to how to make video active Closed (OpenAlex).
“Live or lecture capture: evidence from a classroom RCT,” Int Rev Econ Educ 2022 (SSRN 3933067) Live vs recorded RCT in economics SSRN download blocked. A search summary says recorded did better on the harder topic. Not verified.
Edwards & Clinton 2019, Higher Educ 77:403 (lecture capture, matched cohort) Lecture capture net effect Springer blocked curl. Search snippet says the net effect was negative through lower attendance.
Lindsay & Evans 2021, Math Educ Res J (lecture capture in university maths, systematic review; PMC7882467) Maths-specific Not downloaded. Snippet: a negative association when capture substitutes for attending.
Noetel et al. 2022, RER 92(3), “Multimedia design for learning” (meta-meta-analysis, 29 reviews) Same authors’ later work on how to design video Not opened.
Hew et al. 2021, ERR, review of flipped meta-analysis methodology (green OA) Quality critique Not opened.
Deslauriers, Schelew & Wieman 2011, Science 332 Active learning in a large physics class Closed.
Xu & Jaggars 2014 (CCRC), online course performance gaps by student group Observational; weaker students lose more online Not opened.
Preprint critique of Noetel (preprints.org 202401.0247) Argues demographics were not considered Not opened.
For the sibling search (LMIC/secondary): Bastani 2025 (Turkey; opened above) and Le 2022 (Vietnam, higher ed; opened above) — —