Learning techniques
Part of the AI coach reading library: 49 papers cited in the AI coach notes, each in plain language. Who was studied, how, what it found, why it matters for our students, and where our notes use it. Practice testing, spacing, feedback and the retry, worked examples, problem-first teaching, misconceptions, test preparation and negative marking. Download links open the archived copy and need an Avanti Google login; the Source link is the public original.
The overviews are drawn from the extraction tables and literature reviews written when each paper was read, not from a fresh reading. Where a number here and in a note differ, the note’s section is the one to check.
Practice testing and retrieval
Section titled “Practice testing and retrieval”Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Does testing yourself beat rereading?
Section titled “Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Does testing yourself beat rereading?”Roediger and Karpicke 2006, Psychological Science 17(3). Download · Source
- Who was studied: 120 US undergraduates (Experiment 1) reading prose passages.
- How: Lab experiment. Students either studied a passage and then took a recall test without feedback, or studied it twice. Recall was tested after 5 minutes, 2 days or a week.
- What it found: After a week, tested students recalled 56% of the passage against 42% for those who restudied (d = 0.83; d is an effect size in standard deviations, SD, the usual unit for comparing gains). After 5 minutes the restudy group was ahead, 81% against 75%.
- Why it matters for us: A practice session built on recall will look worse on the day than rereading, so the coach should not judge it by same-day accuracy.
- Where we use it: Meta-analysis section 3.2, as the classic testing result; Learning science literature review, retrieval practice.
The Effect of Testing Versus Restudy on Retention: A Meta-Analytic Review of the Testing Effect. How big is the testing effect across many studies?
Section titled “The Effect of Testing Versus Restudy on Retention: A Meta-Analytic Review of the Testing Effect. How big is the testing effect across many studies?”Rowland 2014, Psychological Bulletin 140(6). Download · Source
- Who was studied: 61 studies, mostly lab studies with US undergraduates learning word lists and prose.
- How: Meta-analysis (a statistical pooling of many studies’ results) of testing against restudying, 159 effect sizes.
- What it found: g = 0.50 (95% confidence interval, CI, 0.42 to 0.58; g is an effect size in SD, like d). Larger with feedback (0.73) than without (0.39), and larger when the final test came a day or more later (0.69 against 0.41). Recognition-type initial tests, like MCQs, gave 0.29 against 0.61 for recall. Published studies gave 0.58 and unpublished ones 0.25.
- Why it matters for us: Recall-style prompts may teach more than MCQ prompts, even though JEE itself is mostly MCQ.
- Where we use it: Meta-analysis section 3.2, as one of the testing meta-analyses; Learning science literature review, retrieval practice and the MCQ-versus-recall disagreement.
Rethinking the Use of Tests: A Meta-Analysis of Practice Testing. Does practice testing work across ages and settings?
Section titled “Rethinking the Use of Tests: A Meta-Analysis of Practice Testing. Does practice testing work across ages and settings?”Adesope, Trevisan and Sundararajan 2017, Review of Educational Research 87(3). Download · Source
- Who was studied: 15,427 learners at all levels, in lab and classroom studies: 272 effect sizes from 118 articles.
- How: Meta-analysis of practice tests against any other condition, including restudying.
- What it found: g = 0.61 (0.58 to 0.65) overall; 0.51 against restudy; 0.83 for secondary students (Grades 7-12, from only 19 effect sizes); 0.67 in classrooms. Feedback made little difference (0.63 with, 0.60 without). MCQ practice gave 0.70 against 0.48 for short answer. Publication bias was checked only with a fail-safe N.
- Why it matters for us: Practice testing works across levels, but its published average is an upper bound and its MCQ finding conflicts with Rowland 2014.
- Where we use it: Meta-analysis section 3.11, as an older meta-analysis whose mean is an upper bound; Learning science literature review, retrieval practice and feedback.
Testing (Quizzing) Boosts Classroom Learning: A Systematic and Meta-Analytic Review. Does quizzing work in real classrooms?
Section titled “Testing (Quizzing) Boosts Classroom Learning: A Systematic and Meta-Analytic Review. Does quizzing work in real classrooms?”Yang, Luo, Vadillo, Yu and Shanks 2021, Psychological Bulletin 147(4). Download · Source
- Who was studied: 48,582 students in 223 classroom studies, from primary school to university.
- How: Meta-analysis of in-class quizzing against any control, 574 effect sizes, with several checks for publication bias.
- What it found: g = 0.50 (0.44 to 0.56), or 0.43 to 0.48 after bias adjustment. Against restudying, 0.33. High school 0.65 (49 effects), maths and statistics 0.43, problem-solving items 0.45. Quizzing with corrective feedback gave 0.54 against 0.37 without.
- Why it matters for us: Quizzing holds up in real classrooms, but the honest gain over rereading is about a third of an SD.
- Where we use it: Meta-analysis section 3.2 and the planning ranges, as an anchor estimate for practice testing on formulas and facts.
A Meta-Analysis of the Effectiveness and Boundary Conditions of Retrieval Practice, Distributed Practice, and Interleaved Practice in Authentic School Classrooms. Do testing, spacing and mixing work in real school classes?
Section titled “A Meta-Analysis of the Effectiveness and Boundary Conditions of Retrieval Practice, Distributed Practice, and Interleaved Practice in Authentic School Classrooms. Do testing, spacing and mixing work in real school classes?”Ruitenburg, Goos, Lambert, Kirschner, Jarodzka and Camp 2026, Educational Psychology Review 38(1). Download · Source
- Who was studied: Primary and secondary students in real school classrooms, 2006-2024, mostly in the US, the Netherlands, Germany, Brazil, Canada and China: 53 studies and 7,303 students in all.
- How: Meta-analysis with three levels, run separately for retrieval practice (31 studies, 3,391 students), spacing (15 studies, 2,010) and interleaving (10 studies, 1,902). The archived copy is an excerpt of the web version.
- What it found: Retrieval g = 0.37 (0.18 to 0.56), 0.52 in secondary school, but -0.12 (-0.61 to 0.38) on procedural knowledge and 0.02 on transfer. Spacing g = 0.20 (0.03 to 0.36), 0.39 against fully massed practice and about zero against slightly less spaced practice. Interleaving g = 0.40 (0.10 to 0.71), 0.54 in maths; 0.03 with feedback after every item against 0.82 with feedback at the end of the set.
- Why it matters for us: It is the closest match to our classrooms, and it says quizzing helps with facts but not with multi-step procedures, which is most of JEE.
- Where we use it: Meta-analysis sections 3.2 and 3.4 and the planning ranges, as the main anchor for practice testing, spacing and interleaving; What works, Summary and section c).
A Meta-Analytic Review of the Effectiveness of Spacing and Retrieval Practice for Mathematics Learning. Do testing and spacing work in maths?
Section titled “A Meta-Analytic Review of the Effectiveness of Spacing and Retrieval Practice for Mathematics Learning. Do testing and spacing work in maths?”Murray, Horner and Göbel 2025, Educational Psychology Review 37(3). Download · Source
- Who was studied: Learners in lab and course studies of maths: 7 studies of testing (32 effects) and 27 studies of spacing (53 effects).
- How: Meta-analysis. The full text is paywalled; the archived copy is the preprint, whose numbers differ slightly from the published abstract.
- What it found: Testing against restudy in maths gave g = 0.18, with a CI that crosses zero (0.22 from 6 studies in the preprint). Spacing gave g = 0.28: 0.24 in course-embedded studies and 0.43 with isolated material, which the authors treat as less dependable.
- Why it matters for us: In maths, the subject closest to JEE problem solving, quizzing yourself adds little that is reliable.
- Where we use it: Meta-analysis section 3.2, for the small testing effect in maths.
Not New, but Nearly Forgotten: The Testing Effect Decreases or Even Disappears as the Complexity of Learning Materials Increases. Does testing help with complex problems?
Section titled “Not New, but Nearly Forgotten: The Testing Effect Decreases or Even Disappears as the Complexity of Learning Materials Increases. Does testing help with complex problems?”van Gog and Sweller 2015, Educational Psychology Review 27. Download · Source
- Who was studied: University, vocational and primary students in the Netherlands and Australia, learning electrical-circuit fault diagnosis, probability and bus timetables.
- How: Narrative review of 8 experiments in which students solved a problem after a worked example, instead of restudying the example.
- What it found: No testing effect in any of the 8 experiments, immediately or about a week later. The authors argue the effect shrinks as the number of interacting elements rises. A reply (Karpicke and Aue 2015) argues for a small positive effect.
- Why it matters for us: For a JEE problem type a student cannot yet do, studying a solution beats trying to recall one.
- Where we use it: Meta-analysis section 3.2, for the limit of practice testing on multi-step problems; Learning science literature review, retrieval practice and the design rules.
The Value of Applied Research: Retrieval Practice Improves Classroom Learning and Recommendations from a Teacher, a Principal, and a Scientist. Does quizzing help in an ordinary school?
Section titled “The Value of Applied Research: Retrieval Practice Improves Classroom Learning and Recommendations from a Teacher, a Principal, and a Scientist. Does quizzing help in an ordinary school?”Agarwal, Bain and Chamberlain 2012, Educational Psychology Review 24. Download · Source
- Who was studied: US middle-school social studies classes, several classes.
- How: Review of classroom experiments in which, within each student, some material was quizzed and some was not.
- What it found: At the end of the semester students got 79% right on quizzed material against 67% on material that was not quizzed.
- Why it matters for us: Low-stakes quizzing in a real school carries over to later unit exams.
- Where we use it: Learning science literature review, retrieval practice.
Retrieval Practice Consistently Benefits Student Learning: A Systematic Review of Applied Research in Schools and Classrooms. How widely has classroom quizzing been tested?
Section titled “Retrieval Practice Consistently Benefits Student Learning: A Systematic Review of Applied Research in Schools and Classrooms. How widely has classroom quizzing been tested?”Agarwal, Nunes and Blunt 2021, Educational Psychology Review 33(4). Download · Source
- Who was studied: 5,374 students in 50 experiments from 37 studies, in school, undergraduate and medical-school classrooms.
- How: Systematic review of retrieval practice against no retrieval, without pooling into one average.
- What it found: 57% of the effect sizes were medium or large. 94% of the studies came from rich Western countries; only 6% of experiments were elsewhere.
- Why it matters for us: Almost none of the classroom evidence on quizzing comes from students like ours.
- Where we use it: Meta-analysis section 3.11, for the population limit of the evidence.
Improving Students’ Learning with Effective Learning Techniques. Which study techniques are worth a student’s time?
Section titled “Improving Students’ Learning with Effective Learning Techniques. Which study techniques are worth a student’s time?”Dunlosky, Rawson, Marsh, Nathan and Willingham 2013, Psychological Science in the Public Interest 14(1). Download · Source
- Who was studied: Hundreds of studies of ten study techniques, mostly lab studies with undergraduates.
- How: Narrative review that rates each technique’s usefulness.
- What it found: Practice testing and spacing rated “high utility”; interleaving, self-explanation and elaborative questioning “moderate”; rereading, highlighting and summarising “low”.
- Why it matters for us: A student’s self-study hours should go to testing and spacing, not to rereading notes.
- Where we use it: Learning science literature review, retrieval practice and interleaving; listed in the Meta-analysis references.
Spacing and mixed practice
Section titled “Spacing and mixed practice”Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention. How long should the gap be before a review?
Section titled “Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention. How long should the gap be before a review?”Cepeda, Vul, Rohrer, Wixted and Pashler 2008, Psychological Science 19(11). Download · Source
- Who was studied: 1,354 adults online, learning obscure trivia facts.
- How: Online experiment. One review session after a gap of up to 3.5 months, then a final test 7, 35, 70 or 350 days later.
- What it found: The best gap was about 20% of the time to the test for horizons of a few weeks, falling to about 5% at a year. For tests 7, 35, 70 and 350 days away, the best gaps for recall were 1, 11, 21 and 21 days. The best gap beat no gap by d = 1.1 on recall.
- Why it matters for us: Reviews should be scheduled back from the JEE date, with gaps that shrink as a share of the time left; the paper did not test problem solving or horizons beyond 350 days.
- Where we use it: Meta-analysis section 3.4, for the shape of a spacing schedule; Learning science literature review, spacing and the Learning Framework corrections.
A Meta-Analytic Review of the Benefit of Spacing out Retrieval Practice Episodes on Retention. Does spacing out quizzes help, and does the schedule shape matter?
Section titled “A Meta-Analytic Review of the Benefit of Spacing out Retrieval Practice Episodes on Retention. Does spacing out quizzes help, and does the schedule shape matter?”Latimier, Peyre and Ramus 2021, Educational Psychology Review 33(3). Download · Source
- Who was studied: Mostly undergraduates; 9 of the 11 studies were in the lab. Materials included word pairs, prose and maths problems.
- How: Meta-analysis of spaced against massed retrieval practice, 39 effects from 11 studies. The archived copy is the preprint.
- What it found: Spaced retrieval beat massed retrieval by g = 1.02 (0.68 to 1.36), or 0.74 after trim-and-fill (a correction that adds the studies publication bias may have hidden). An expanding schedule did no better than an even one (g = 0.03, -0.10 to 0.17).
- Why it matters for us: The coach can use a simple even schedule; there is no gain from a clever expanding one.
- Where we use it: Meta-analysis section 3.4, for expanding against even schedules.
Single-Paper Meta-Analyses of the Effects of Spaced Retrieval Practice in Nine Introductory STEM Courses: Is the Glass Half Full or Half Empty? Does spacing work in real college science courses?
Section titled “Single-Paper Meta-Analyses of the Effects of Spaced Retrieval Practice in Nine Introductory STEM Courses: Is the Glass Half Full or Half Empty? Does spacing work in real college science courses?”Bego, Lyle et al. 2024, International Journal of STEM Education 11. Download · Source
- Who was studied: Introductory STEM undergraduates at the University of Louisville, US: 578 students in eight courses and 180 in calculus. Courses included biology, chemistry, physics and engineering.
- How: Within each student, quiz questions were randomly assigned to be spaced over three fortnightly quizzes or massed on one. A test 30-34 days after the last quiz measured the result. In eight courses the quizzes showed only right or wrong. The archived copy is verbatim excerpts.
- What it found: A pooled gain of 2.06 percentage points (0.16 to 3.97), but only 1.50 (-0.18 to 3.26, not significant) without calculus. Spacing helped significantly in 2 of 9 courses; physics went the wrong way (g = -0.12).
- Why it matters for us: Spacing in real science courses is fragile, and may need informative feedback to work.
- Where we use it: Meta-analysis section 3.4, for spacing in real STEM courses; Learning science literature review, spacing and the design rules.
Similarity Matters: A Meta-Analysis of Interleaved Learning and Its Moderators. How much does mixing problem types help?
Section titled “Similarity Matters: A Meta-Analysis of Interleaved Learning and Its Moderators. How much does mixing problem types help?”Brunmair and Richter 2019, Psychological Bulletin 145(11). Download · Source
- Who was studied: Mostly adults in the lab, learning paintings, photos, maths, texts and words: 238 effects from 158 samples in 59 papers.
- How: Meta-analysis of interleaved (mixed) against blocked practice.
- What it found: g = 0.42 (0.34 to 0.50) overall, 0.29 after trim-and-fill. Maths 0.34 (0.11 to 0.57, 45 effects), with no sign of bias in that subset. Paintings 0.67; word lists -0.39. The delay before the test did not change the effect.
- Why it matters for us: Mixing helps in maths at about a third of an SD, smaller than the headline classroom trials.
- Where we use it: Meta-analysis section 3.4 and the planning ranges, as the maths estimate for interleaving.
Interleaved Practice Improves Mathematics Learning. Does mixing problem types help Grade 7 maths?
Section titled “Interleaved Practice Improves Mathematics Learning. Does mixing problem types help Grade 7 maths?”Rohrer, Dedrick and Stershic 2015, Journal of Educational Psychology. Download · Source
- Who was studied: 126 US Grade 7 maths students.
- How: Classroom experiment over about three months: interleaved against blocked practice, then an unannounced test.
- What it found: d = 0.42 on a test 1 day later and 0.79 on a test 30 days later.
- Why it matters for us: Mixed practice trains students to pick the method, the skill a mixed JEE paper tests.
- Where we use it: Learning science literature review, interleaving and the design rules.
A Randomized Controlled Trial of Interleaved Mathematics Practice. Does mixed homework raise maths scores across many classes?
Section titled “A Randomized Controlled Trial of Interleaved Mathematics Practice. Does mixed homework raise maths scores across many classes?”Rohrer, Dedrick, Hartwig and Cheung 2020, Journal of Educational Psychology 112(1). Download · Source
- Who was studied: 787 Grade 7 students in 54 maths classes taught by 15 teachers in Florida, US.
- How: Randomised controlled trial (RCT: classes assigned to each arm by lottery). Over four months, classes got mostly mixed or mostly blocked homework, then an unannounced 16-item test a month later.
- What it found: 61% against 38% (d = 0.83, 0.68 to 0.97; this interval ignores that students were randomised in classes). Effects by teacher ranged from d = 0.23 to 1.48. Mixed assignments took students longer, and mixing also spaces practice, so the two cannot be separated.
- Why it matters for us: It is the strongest trial of mixed practice, but on Grade 7 maths and a targeted test, not a full exam.
- Where we use it: Meta-analysis section 3.4, for the size of the classroom effect; Learning science literature review, interleaving.
Interleaved Practice Enhances Memory and Problem-Solving Ability in Undergraduate Physics. Does mixing physics homework reach the real exam?
Section titled “Interleaved Practice Enhances Memory and Problem-Solving Ability in Undergraduate Physics. Does mixing physics homework reach the real exam?”Samani and Pan 2021, npj Science of Learning 6. Download · Source
- Who was studied: About 290 introductory physics undergraduates in two lecture sections, US.
- How: Classroom crossover over 8 weeks: the two sections swapped mixed and blocked homework, then sat surprise tests of new, harder problems.
- What it found: On the surprise tests, d = 0.40 and 0.91. On the course midterms three days later, d = 0.20 (-0.04 to 0.43) and 0.02 (-0.21 to 0.25), neither significant. Only two class groups were compared.
- Why it matters for us: Gains from mixed practice on a researcher’s test may not show up on the exam that counts.
- Where we use it: Meta-analysis section 3.4, for why mixed practice is held back for JEE.
The Long-Term Distributional Impacts of a Full-Year Interleaving Math Program in Nigeria. Does a year of mixed practice help everyone?
Section titled “The Long-Term Distributional Impacts of a Full-Year Interleaving Math Program in Nigeria. Does a year of mixed practice help everyone?”van der Haar, Gray-Lobe, Kremer and de Laat 2023, EdWorkingPaper 23-876. Download · Source
- Who was studied: Grade 5 pupils in 62 classrooms at 58 low-cost private schools in Lagos, Nigeria: 665 on term tests, 518 on the year-end test.
- How: RCT by classroom, a full year of mixed against blocked problem sets.
- What it found: +0.29 SD (0.05 to 0.53) on end-of-term tests, but +0.04 SD on the cumulative year-end test (about -0.27 to 0.35). Gains at the bottom of the class were offset by some losses at the top; the authors’ explanation for the top is labelled speculative.
- Why it matters for us: Mixed practice may not help, and may hurt, the strongest students on a long cumulative exam, so effects should be measured by ability band.
- Where we use it: Meta-analysis section 3.4, for why mixed practice is held back for JEE; What works, section c); Learning science literature review, interleaving and the design rules.
Feedback, the retry and mastery
Section titled “Feedback, the retry and mastery”The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research. Which kinds of feedback help most?
Section titled “The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research. Which kinds of feedback help most?”Wisniewski, Zierer and Hattie 2020, Frontiers in Psychology 10. Download · Source
- Who was studied: About 61,000 learners at all levels: 959 effects from 435 studies, drawn from 32 earlier Hattie meta-analyses.
- How: Meta-analysis of feedback against no feedback or before-and-after designs, broken down by type of feedback.
- What it found: d = 0.48 (0.44 to 0.51) overall; 0.42 in controlled designs against 0.63 in before-and-after ones. “High-information” feedback gave 0.99 from 42 effects, corrective feedback 0.46, and reward or punishment 0.24. The high-information category adds self-regulation advice, and the corrective one already includes the correct answer. Publication bias was detected.
- Why it matters for us: The often-quoted 0.99 against 0.46 does not compare like with like, so the coach’s feedback design should rest on direct comparisons instead.
- Where we use it: Meta-analysis section 3.5, as the comparison not to quote; Learning science literature review, feedback.
Effects of Computer-Based Feedback on Lower- and Higher-Order Learning Outcomes: A Network Meta-Analysis. Is a cross, the correct answer, or an explanation best?
Section titled “Effects of Computer-Based Feedback on Lower- and Higher-Order Learning Outcomes: A Network Meta-Analysis. Is a cross, the correct answer, or an explanation best?”Mertens, Finn and Lindner 2022, Journal of Educational Psychology 114(8). Download · Source
- Who was studied: 77 randomised experiments, 163 effects; 58% undergraduates and 21% middle school; a third of effects in maths and a third in science.
- How: Network meta-analysis (it compares several feedback types at once against no feedback), randomised experiments only. The archived copy is an excerpt of the web version.
- What it found: On near items: right or wrong only 0.24, the correct answer 0.49, an explanation 0.71. On transfer items: 0.14, 0.22 and 0.46. In maths transfer items (12 comparisons) the plain correct answer did better (0.57) than an explanation (0.18, not significant).
- Why it matters for us: A cross alone does almost nothing; showing the correct answer and working gives most of the value, and an explanation adds about 0.15 to 0.2 SD.
- Where we use it: Meta-analysis section 3.5 and the planning ranges, as the anchor for feedback type; What works, section d).
Two-Sigma Tutoring: Separating Science Fiction from Science Fact. Where did the famous two-sigma tutoring effect come from?
Section titled “Two-Sigma Tutoring: Separating Science Fiction from Science Fact. Where did the famous two-sigma tutoring effect come from?”von Hippel 2024, Education Next 24(2). Download · Source
- Who was studied: The two small dissertation trials behind Bloom’s 1984 claim: US Grades 4, 5 and 8, learning probability and map-reading.
- How: Re-reading of the original dissertations.
- What it found: The trials lasted three weeks, on topics chosen because they were unfamiliar, with tests tied to what was taught. Tutored students also got extra testing, feedback and correction, then a second quiz with new questions; the author attributes about half of the two-sigma effect to this.
- Why it matters for us: The part worth copying is the retry on new questions after feedback, not the two-sigma size.
- Where we use it: Meta-analysis section 3.6, as the basis for the unaided retry; What works, Summary and section d); Learning science literature review, mastery learning and the design rules.
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Can group teaching match a personal tutor?
Section titled “The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Can group teaching match a personal tutor?”Bloom 1984, Educational Researcher 13(6). Download · Source
- Who was studied: Small samples of school students in two PhD experiments.
- How: Essay summarising those experiments.
- What it found: Tutored students scored about 2 SD above a conventional class, and mastery-learning students about 1 SD above.
- Why it matters for us: It is the origin of the two-sigma claim, which should not be used as a target for our students (see von Hippel 2024).
- Where we use it: Learning science literature review, mastery learning and the design rules; mentioned in Meta-analysis sections 3.6 and 3.11 as the claim the coach should not expect.
Mastery Learning Reconsidered. Does mastery learning raise scores on standardised tests?
Section titled “Mastery Learning Reconsidered. Does mastery learning raise scores on standardised tests?”Slavin 1987, Report No. 7, Center for Research on Elementary and Middle Schools. Download · Source
- Who was studied: School students in studies of group-based mastery learning lasting four weeks or more.
- How: Best-evidence synthesis (a review that keeps only the better-designed studies).
- What it found: “Essentially no evidence” of effects on standardised tests, and moderate positive effects on tests the experimenters wrote.
- Why it matters for us: A mastery gate can raise scores on its own items without moving mock scores, so it should be judged on mocks.
- Where we use it: Learning science literature review, mastery learning, the design rules and the spec conflicts.
Interpreting Effect Sizes of Education Interventions. What counts as a big effect in a school programme?
Section titled “Interpreting Effect Sizes of Education Interventions. What counts as a big effect in a school programme?”Kraft 2020, Educational Researcher (EdWorkingPaper 19-10 copy). Download · Source
- Who was studied: US school students, through about 750 RCTs with standardised achievement tests.
- How: Methods paper that sets benchmarks from the spread of effects in those trials.
- What it found: On standardised tests, under 0.05 SD is small, 0.05 to under 0.20 SD medium, and 0.20 SD or more large. Tests designed by the researchers give larger effects.
- Why it matters for us: A pilot that moves mock scores by 0.10 SD would be a real result for a school programme.
- Where we use it: Meta-analysis Appendix B, for the benchmarks; Learning science literature review, calibrating expected effects and the design rules; India context literature review, for the evaluation design (design rule 11).
Worked, faded and erroneous examples
Section titled “Worked, faded and erroneous examples”A Meta-Analysis of the Worked Examples Effect on Mathematics Performance. Do solved examples beat solving problems in maths?
Section titled “A Meta-Analysis of the Worked Examples Effect on Mathematics Performance. Do solved examples beat solving problems in maths?”Barbieri, Miller-Cotto, Clerjuste and Chawla 2023, Educational Psychology Review 35. Download · Source
- Who was studied: Primary to college students, mostly middle school and up (only 5 primary studies): 55 studies and 181 effects from 43 articles, 23 from the US and 20 from elsewhere. Maths only.
- How: Meta-analysis of 53 RCTs and 2 quasi-experiments comparing worked-example conditions with problem solving.
- What it found: g = 0.48 (0.36 to 0.60), 0.44 after trim-and-fill. The spread between studies was extreme (I² = 94%, the share of variation that is real difference between studies), so a new study could land anywhere from about -1 to +2. Correct-only examples did better than incorrect or mixed ones (+0.26); adding self-explanation prompts went with smaller effects (-0.24).
- Why it matters for us: Solved examples are the best-evidenced help for a student weak in a chapter, but the size in any one setting is uncertain.
- Where we use it: Meta-analysis sections 3.2 and 3.5 and the planning ranges, as the anchor for worked examples; What works, Summary and section b).
Learning Algebra by Example in Real-World Classrooms. Do worked-example assignments help in ordinary algebra classes?
Section titled “Learning Algebra by Example in Real-World Classrooms. Do worked-example assignments help in ordinary algebra classes?”Booth, Oyer, Paré-Blagoev, Elliot, Barbieri, Augustine and Koedinger 2015, Journal of Research on Educational Effectiveness 8(4). Download · Source
- Who was studied: Non-honours Algebra I high-school students in five US districts: 425 in the main trial and 51 in a three-class trial.
- How: RCTs within classrooms over one unit. Assignments swapped some practice problems for correct and incorrect worked examples with explanation prompts.
- What it found: In the main trial, end-of-unit scores rose by 0.10 SD (-0.06 to 0.26), too small to rule out zero; students who started with less prior knowledge gained more. The small three-class trial gave about d = 0.67.
- Why it matters for us: In real classrooms worked examples give small gains, mostly for weaker students.
- Where we use it: Meta-analysis section 3.2, for the smaller classroom effect.
The Efficiency of Worked Examples Compared to Erroneous Examples, Tutored Problem Solving, and Problem Solving in Computer-Based Learning Environments. Which way of practising chemistry is quickest?
Section titled “The Efficiency of Worked Examples Compared to Erroneous Examples, Tutored Problem Solving, and Problem Solving in Computer-Based Learning Environments. Which way of practising chemistry is quickest?”McLaren, van Gog, Ganoe, Karabinos and Yaron 2016, Computers in Human Behavior 55. Download · Source
- Who was studied: US Grade 10-11 chemistry students learning stoichiometry: about 155 and 116 in two trials.
- How: Two classroom RCTs of six 40-minute sessions in a web tutor: worked examples, erroneous examples, tutored problems or plain problem solving.
- What it found: All four produced the same learning. Worked examples took 46-69% less time and less mental effort.
- Why it matters for us: For a student short of hours before JEE, the dependable gain from worked examples is time saved.
- Where we use it: Meta-analysis sections 3.2 and 3.5, for the time saving; What works, section b).
Accounting for Beneficial Effects of Worked Examples in Tutored Problem Solving. Do worked steps help inside tutoring software?
Section titled “Accounting for Beneficial Effects of Worked Examples in Tutored Problem Solving. Do worked steps help inside tutoring software?”Salden, Koedinger, Renkl, Aleven and McLaren 2010, Educational Psychology Review 22. Download · Source
- Who was studied: High-school geometry, algebra and chemistry students using Cognitive Tutor software in the US and Germany, including 4 geometry studies.
- How: Narrative review of lab and classroom studies that added worked steps to the tutor, faded either for every student alike or adapted to the quality of each student’s self-explanations.
- What it found: Fading adapted to the student beat fixed fading and plain tutored problem solving on delayed retention. Worked examples cut time by 19-26% for equal learning. No effect sizes reported.
- Why it matters for us: Removing worked steps as each student shows the step supports a coach that fades help student by student.
- Where we use it: Meta-analysis section 3.3, for adaptive fading; What works, section b).
Using Example Problems to Improve Student Learning in Algebra: Differentiating Between Correct and Incorrect Examples. Do wrong worked examples teach as well as right ones?
Section titled “Using Example Problems to Improve Student Learning in Algebra: Differentiating Between Correct and Incorrect Examples. Do wrong worked examples teach as well as right ones?”Booth, Lange, Koedinger and Newton 2013, Learning and Instruction 25. Download · Source
- Who was studied: US high-school and middle-school Algebra I students using Cognitive Tutor software: about 115 and 67 analysed in two trials.
- How: Two classroom RCTs. Students studied correct, incorrect or both kinds of worked examples, with self-explanation prompts given as sentence-building menus, against the tutor alone (trial 1) or against each other (trial 2).
- What it found: Trial 1: examples raised conceptual scores, 63% against 58% (about d = 0.46), though example students solved 8 fewer practice problems. Trial 2: incorrect or combined examples beat correct-only examples on conceptual features. Procedural scores did not differ.
- Why it matters for us: Showing a wrong worked solution, with a constrained “why” prompt, can build understanding without hurting procedure.
- Where we use it: Meta-analysis section 3.5, for self-explanation menus that worked in classrooms.
Delayed Learning Effects with Erroneous Examples: A Study of Learning Decimals with a Web-Based Tutor. Does finding and fixing a planted error help later?
Section titled “Delayed Learning Effects with Erroneous Examples: A Study of Learning Decimals with a Web-Based Tutor. Does finding and fixing a planted error help later?”McLaren, Adams and Mayer 2015, International Journal of Artificial Intelligence in Education 25(4). Download · Source
- Who was studied: 390 Grade 6 students in Pittsburgh, US, learning decimals.
- How: RCT in a web tutor. Students found, explained and fixed errors in a fictitious student’s work, or solved problems with support and explanation.
- What it found: d = 0.33 on a delayed test a week after the immediate one, against 0.22 on the immediate test (p = .055). Problem-solvers liked their version more.
- Why it matters for us: Erroneous examples help a little, and more on delayed tests; no study used the student’s own wrong answer as the example, which is what the coach would do.
- Where we use it: Meta-analysis section 3.5, for erroneous examples.
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction. Do worked examples stop helping once a student knows the topic?
Section titled “Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction. Do worked examples stop helping once a student knows the topic?”Kalyuga 2007, Educational Psychology Review 19. Download · Source
- Who was studied: School and adult learners in the experiments it reviews, including high-school students in Australia and engineering students in the US; 42 to 124 per experiment in the tabulated studies.
- How: Review of expertise-reversal experiments.
- What it found: In high-school coordinate geometry, worked examples beat problem solving for novices (d = 1.20) and lost for more knowledgeable students (d = -0.36). In engineering, novices did better with slow fading and knowledgeable students with fast fading. A one-question “what is your first step?” test correlated up to .92 with longer tests. Linear-equations tutorials that used it to choose how much to show beat fixed versions (effect sizes 0.46 and 0.55).
- Why it matters for us: The same student can need worked examples in one chapter and straight problems in another, and a first-step probe can tell which.
- Where we use it: Meta-analysis section 3.7, for the first-step probe; Learning science, Step-based tutoring and Lesson design literature reviews, for routing by prior knowledge and fading.
A Cornerstone of Adaptivity: A Meta-Analysis of the Expertise Reversal Effect. How much do weak and strong students differ in what helps them?
Section titled “A Cornerstone of Adaptivity: A Meta-Analysis of the Expertise Reversal Effect. How much do weak and strong students differ in what helps them?”Tetzlaff, Simonsmeier, Peters and Brod 2025, Learning and Instruction 98. Download · Source
- Who was studied: 5,924 learners in 60 studies (88 effects), primary to adult, including 18 secondary-school studies; 38 of the 60 studies in STEM.
- How: Meta-analysis comparing more help (worked examples, guidance) with less help, separately for learners with low and high prior knowledge.
- What it found: More help raised scores for low prior knowledge (d = 0.51, 0.26 to 0.75) and lowered them for high prior knowledge (d = -0.43, -0.65 to -0.21). When prior knowledge was measured by a test, the cost for strong students was about -0.21.
- Why it matters for us: Withholding an example from a weak student costs more than giving one to a strong student, so when the coach is unsure it should show it.
- Where we use it: Meta-analysis section 3.7 and the planning ranges, as the anchor for worked examples with strong students; What works, section b).
Problem first, discovery, contrasting cases and misconceptions
Section titled “Problem first, discovery, contrasting cases and misconceptions”When Problem Solving Followed by Instruction Works: Evidence for Productive Failure. Should students attempt a problem before being taught?
Section titled “When Problem Solving Followed by Instruction Works: Evidence for Productive Failure. Should students attempt a problem before being taught?”Sinha and Kapur 2021, Review of Educational Research 91(5). Download · Source
- Who was studied: Grade 2 to adult learners, 45% of comparisons in Grades 6 to 10; mostly maths and science; Asia, North America, Europe and Australia. 53 studies, 166 comparisons.
- How: Meta-analysis of problem solving then instruction (“productive failure”) against instruction then problem solving.
- What it found: On concepts and transfer g = 0.36 (0.20 to 0.51), up to 0.37-0.58 when designs followed productive-failure principles closely. On procedures g = -0.03 (-0.20 to 0.15). In Grades 2 to 5 teaching first did better. The authors’ “bias-corrected” 0.87 is a p-curve estimate that moves upward and should not be quoted.
- Why it matters for us: Attempting first can help understanding of ideas, but does nothing for procedures.
- Where we use it: Meta-analysis section 3.1 and the planning ranges, as the anchor for problem-first; Lesson design literature review, struggle first.
Does Discovery-Based Instruction Enhance Learning? Can students work ideas out for themselves?
Section titled “Does Discovery-Based Instruction Enhance Learning? Can students work ideas out for themselves?”Alfieri, Brooks, Aldrich and Tenenbaum 2011, Journal of Educational Psychology 103(1). Download · Source
- Who was studied: Children, adolescents and adults in maths, science and other subjects: 5,226 learners in 108 studies (unassisted discovery) and 4,243 in 56 studies (guided discovery).
- How: Two meta-analyses: unassisted discovery against explicit teaching, and discovery with guidance against other teaching.
- What it found: Unassisted discovery lost to explicit teaching, d = -0.38 (-0.50 to -0.25), and -0.53 for adolescents. Discovery with guidance won, d = 0.30 (0.15 to 0.44): guided discovery 0.50, asking for explanations 0.36, but generating answers alone -0.15, and only 0.11 in science. Against worked examples it made no difference (0.06).
- Why it matters for us: Students should never be left to discover a chapter’s ideas alone; self-study needs explanation built in.
- Where we use it: Meta-analysis section 3.1 and the planning ranges, as the anchor for teaching first; What works, Summary and section a); Lesson design literature review.
Aligning the Structural Components Across Learning Tasks of Case Comparisons. Does comparing two cases side by side help learning?
Section titled “Aligning the Structural Components Across Learning Tasks of Case Comparisons. Does comparing two cases side by side help learning?”Alfieri, Nokes and Schunn 2011, SREE Fall 2011 conference abstract. Download · Source
- Who was studied: Learners in 57 experiments (336 tests), mostly adults (41 of 57), mostly in the US and Europe; maths, science and other subjects.
- How: Meta-analysis of comparing cases against studying one case at a time, no study, or ordinary teaching. This is a conference abstract; the 2013 journal version was not opened.
- What it found: d = 0.60 (0.47 to 0.72) overall; 0.37 against studying the same cases one after the other. For adolescents, 0.34 (-0.01 to 0.68) from only 4 studies. Maths 0.23, science 0.72.
- Why it matters for us: Contrasting cases are promising for teenagers but rest on very few studies.
- Where we use it: Meta-analysis section 3.8 and the planning ranges, as the estimate for contrasting cases.
Practicing Versus Inventing with Contrasting Cases: The Effects of Telling First on Learning and Transfer. Should students invent a formula before being told it?
Section titled “Practicing Versus Inventing with Contrasting Cases: The Effects of Telling First on Learning and Transfer. Should students invent a formula before being told it?”Schwartz, Chase, Oppezzo and Chin 2011, Journal of Educational Psychology 103(4). Download · Source
- Who was studied: US Grade 8 physics students learning density and speed: 128 in four classes (Experiment 1) and 120 (Experiment 2).
- How: Classroom experiments. Students invented a formula from contrasting cases and were then told the standard one, or were told first and then practised on the same cases.
- What it found: On transfer to a new ratio concept after 21 days, 53.5% against 23.0% (reported d = 0.33), replicated at 50.9% against 23.3%. Word-problem scores were level (70.1% against 75.2%, not significant). Invent-first students were less accurate during the lessons (84.7% against 91.8%), and it cost about 10 extra minutes over 4 days.
- Why it matters for us: Inventing first then being told helps transfer while looking worse during the lesson, so the coach should not judge a session by in-lesson accuracy.
- Where we use it: Meta-analysis section 3.8, for contrasting cases; Lesson design literature review, struggle first and the design rules.
Refutation Text Facilitates Learning: A Meta-Analysis of Between-Subjects Experiments. Does naming and refuting a misconception help?
Section titled “Refutation Text Facilitates Learning: A Meta-Analysis of Between-Subjects Experiments. Does naming and refuting a misconception help?”Schroeder and Kucera 2022, Educational Psychology Review 34(2). Download · Source
- Who was studied: 3,869 learners in 44 comparisons, mostly post-secondary (30 of 44); mostly science; Europe and North America.
- How: Meta-analysis of texts that state and refute a misconception against other texts, mostly standard explanations.
- What it found: g = 0.41 (0.30 to 0.51), 0.28 after trim-and-fill. Secondary school 0.49 (0.10 to 0.88) from only 4 comparisons. Dissertations 0.11 (not significant) against 0.45 in journals. The delay before the test did not change the effect.
- Why it matters for us: Naming the wrong idea and showing where it fails helps, by roughly a quarter to two-fifths of an SD.
- Where we use it: Meta-analysis section 3.8 and the planning ranges, as the anchor for refutation; What works, Summary and section a); Learning science literature review, refutation.
Understanding Misconceptions: Teaching and Learning in Middle School Physical Science. Does knowing students’ favourite wrong answer make a better teacher?
Section titled “Understanding Misconceptions: Teaching and Learning in Middle School Physical Science. Does knowing students’ favourite wrong answer make a better teacher?”Sadler and Sonnert 2016, American Educator, Spring 2016. Download · Source
- Who was studied: 9,556 US Grade 7-8 physical-science students and their 181 volunteer teachers.
- How: Observational. Teachers and students took the same multiple-choice items; classroom gains over a school year were compared by what the teacher knew.
- What it found: On items with a popular wrong answer, classes of teachers who knew both the right answer and the most common wrong one gained 0.56 SD of the pretest, against 0.40 for teachers who knew only the right answer. The link appeared among higher-achieving students; low achievers gained little on these items whatever the teacher knew.
- Why it matters for us: The popular wrong option is a useful signal for the teacher, though this design cannot show cause.
- Where we use it: Meta-analysis section 3.8, for the wrong option as a class-level signal; Learning science literature review, refutation and the design rules.
Psychometric Models of Student Conceptions in Science: Reconciling Qualitative Studies and Distractor-Driven Assessment Instruments. Who picks the misconception option?
Section titled “Psychometric Models of Student Conceptions in Science: Reconciling Qualitative Studies and Distractor-Driven Assessment Instruments. Who picks the misconception option?”Sadler 1998, Journal of Research in Science Teaching 35(3). Download · Source
- Who was studied: 1,250 US Grade 8-12 students at the start and end of an astronomy course.
- How: Item response modelling of a test whose wrong options are common misconceptions, tracing how likely each option is at each ability level.
- What it found: Misconception options were most attractive to middle-ability students. Low-ability students’ wrong answers were mostly guesses. On one item the chance of a correct answer was 0.63 at the lowest ability, dipping to 0.46 at moderate ability.
- Why it matters for us: A weak student’s wrong answer is often a guess, not a misconception, so the coach should check before naming one.
- Where we use it: Meta-analysis section 3.8, for why one wrong answer is a noisy signal.
Exploring the Structure of Misconceptions in the Force Concept Inventory with Modified Module Analysis. Do wrong answers on a physics test cluster into clear misconceptions?
Section titled “Exploring the Structure of Misconceptions in the Force Concept Inventory with Modified Module Analysis. Do wrong answers on a physics test cluster into clear misconceptions?”Wells, Henderson, Stewart, Stewart, Yang and Traxler 2019, Physical Review Physics Education Research 15(2). Download · Source
- Who was studied: US university physics students: 9,225 Force Concept Inventory tests (4,509 before teaching, 4,716 after).
- How: Network analysis of which wrong options students choose together.
- What it found: Wrong options correlated only weakly with each other. An earlier analysis was not replicated. The 9 groups before teaching and 11 after mostly matched known misconceptions, and some were artefacts of how the items are grouped.
- Why it matters for us: One student’s pattern of wrong answers is a weak guide to a single underlying misconception.
- Where we use it: Meta-analysis section 3.8, for why one wrong answer is a noisy signal; Lesson design literature review.
A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptions in Science. Can a multiple-choice test tell a misconception from a guess?
Section titled “A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptions in Science. Can a multiple-choice test tell a misconception from a guess?”Kaltakci Gurel, Eryilmaz and McDermott 2015, EURASIA Journal of Mathematics, Science and Technology Education 11(5). Download · Source
- Who was studied: 273 articles on tests used to diagnose science misconceptions.
- How: Document analysis comparing interviews, open-ended tests, plain multiple-choice tests and multi-tier tests.
- What it found: Plain and two-tier multiple-choice tests cannot separate a misconception from a lack of knowledge or a guess. Adding a confidence rating (three- and four-tier tests) is the proposed fix.
- Why it matters for us: A quick “how sure were you?” should come before the coach names a misconception.
- Where we use it: Meta-analysis section 3.8, for the confidence check; What works, section a).
Combining Peer Discussion with Instructor Explanation Increases Student Learning from In-Class Concept Questions. Is peer discussion, an explanation, or both best after a concept question?
Section titled “Combining Peer Discussion with Instructor Explanation Increases Student Learning from In-Class Concept Questions. Is peer discussion, an explanation, or both best after a concept question?”Smith, Wood, Krauter and Knight 2011, CBE Life Sciences Education 10(1). Download · Source
- Who was studied: US undergraduate genetics students, majors and non-majors, in one course.
- How: Experiment within the course. After answering alone, students got peer discussion only, instructor explanation only, or both, with each question pair randomly assigned; learning was measured on a matched second question.
- What it found: Discussion plus explanation beat either alone (the values are in a figure and were not extracted). Strong students were not helped by explanation alone, and weak non-majors gained little beyond the explanation.
- Why it matters for us: The gain comes from discussion and explanation together, and a one-to-one coach has no peers.
- Where we use it: Meta-analysis section 3.8, for the commit-then-discuss step.
Research-Based Implementation of Peer Instruction: A Literature Review. Does committing to an answer first help on its own?
Section titled “Research-Based Implementation of Peer Instruction: A Literature Review. Does committing to an answer first help on its own?”Vickrey, Rosploch, Rahmanian, Pilarz and Stains 2015, CBE Life Sciences Education 14(1). Download · Source
- Who was studied: Undergraduate STEM courses using Peer Instruction, mostly in the US: physics, biology, chemistry and computer science.
- How: Narrative literature review of each part of Peer Instruction.
- What it found: Studies of the step where each student commits to an answer alone do not report learning gains for that step on its own. No effect sizes.
- Why it matters for us: Asking the student to commit before the coach explains has no evidence of its own.
- Where we use it: Meta-analysis section 3.8, for the commit step.
Test preparation and negative marking
Section titled “Test preparation and negative marking”The Impact of Test Preparation on Performance of Large-Scale Educational Tests: A Meta-Analysis of Experimental Studies. How much does test preparation raise scores?
Section titled “The Impact of Test Preparation on Performance of Large-Scale Educational Tests: A Meta-Analysis of Experimental Studies. How much does test preparation raise scores?”Hao, Baird, El Masri and Double 2026, Review of Educational Research 96(5). Download · Source
- Who was studied: About 16,700 school and university students in 28 experimental and quasi-experimental studies (87 effects) of preparation for large-scale tests.
- How: Meta-analysis of test preparation against none or other preparation.
- What it found: g = 0.26 (0.10 to 0.42), the same in low risk-of-bias studies. Preparation that taught test-taking skills (guessing, eliminating options, managing time) gave 0.35 against 0.05 without; workbooks 0.40 against 0.15. Contact hours made no difference. On college admission tests, 0.14, not significant. The skills comparison is across studies, so it may be confounded.
- Why it matters for us: It is the closest pooled evidence to JEE preparation, and it puts the value in specific test-taking skills, not more hours.
- Where we use it: Meta-analysis section 3.12, for teaching the attempt rule; What works, Summary.
Gender Differences in Willingness to Guess. Do women skip more questions when wrong answers cost marks?
Section titled “Gender Differences in Willingness to Guess. Do women skip more questions when wrong answers cost marks?”Baldiga 2014, Management Science (working-paper copy). Download · Source
- Who was studied: 406 people from the Harvard Business School lab pool, US, answering 20 SAT II history questions with four options.
- How: Lab experiment with random assignment to a small penalty for wrong answers or none.
- What it found: Under the penalty women skipped 2.84 of 20 questions against 1.57 for men, with no gap in knowledge or confidence. With no penalty nobody skipped. Skippers scored lower given what they knew.
- Why it matters for us: Some students, girls in particular, may skip questions they should attempt under JEE’s +4/-1 marking.
- Where we use it: Meta-analysis section 3.12, for who skips; Learning science literature review, negative marking and the design rules.
The Impact of Penalties for Wrong Answers on the Gender Gap in Test Scores. What happened when Chile dropped its penalty?
Section titled “The Impact of Penalties for Wrong Answers on the Gender Gap in Test Scores. What happened when Chile dropped its penalty?”Coffman and Klinowski 2020, PNAS 117(16). Download · Source
- Who was studied: All takers of Chile’s university entrance test, 2013-2016: about 3 million test records.
- How: Natural experiment. In 2015 Chile removed its -0.25 penalty on five-option questions; skips and scores were compared before and after, by gender.
- What it found: Before the change women skipped 1.96 more questions of 80. The gender gap in skipping fell 70% (79% in the top fifth), but the score gap fell only 0.025 SD overall and 0.024 SD in the top fifth.
- Why it matters for us: Skipping too much costs marks, but the score cost is small, so an attempt rule is worth teaching without expecting large gains.
- Where we use it: Meta-analysis section 3.12, for the cost of skipping.
Hit or Miss? Test Taking Behavior in Multiple Choice Exams. Who is too cautious about guessing on an entrance exam?
Section titled “Hit or Miss? Test Taking Behavior in Multiple Choice Exams. Who is too cautious about guessing on an entrance exam?”Akyol, Key and Krishna 2016, NBER Working Paper 22401. Download · Source
- Who was studied: Takers of the Turkish university entrance exam (five options, +1 / -0.25).
- How: A model of guessing behaviour fitted to the exam records, then used to simulate the exam without a penalty.
- What it found: Women and high-ability students were more risk averse. At median ability, women answered when at least 26% sure against 25% for men. Removing the penalty would raise women’s share of the top 5% from 39.9% to 40.2%. The authors favour keeping negative marking.
- Why it matters for us: Strong students may skip answerable questions; JEE makes a blind guess worth +0.25, so skipping costs more here than in Turkey.
- Where we use it: Meta-analysis section 3.12, for the cost of skipping; Learning science literature review, negative marking and the spec conflicts.
Do Time-Constraints Matter? How, Why, and for Whom? Do boys and girls differ in risk-taking under time pressure?
Section titled “Do Time-Constraints Matter? How, Why, and for Whom? Do boys and girls differ in risk-taking under time pressure?”Krishna, Akyol and Ozer 2025, NBER Working Paper 33679. Download · Source
- Who was studied: 83 Grade 12 students (about 41% girls) at one high school in Turkey, taking a 48-question five-option test with a -0.25 penalty.
- How: Experiment in the field: a 40-minute or 75-minute time limit, with a model of risk-taking fitted to the answers.
- What it found: Risk aversion did not differ significantly by gender. Extra time helped boys more. The study is too small to detect a gender difference in risk aversion.
- Why it matters for us: The gender-and-guessing finding is contested, so it should be checked in Avanti mocks before targeting girls.
- Where we use it: Meta-analysis section 3.12, as the weak counter-evidence; Learning science literature review, negative marking.
Gender Differences in Strategic Behaviour under Competitive Pressure: Evidence on Omission Patterns in University Entrance Examinations. Do women answer too few questions on a real entrance exam?
Section titled “Gender Differences in Strategic Behaviour under Competitive Pressure: Evidence on Omission Patterns in University Entrance Examinations. Do women answer too few questions on a real entrance exam?”Pekkarinen 2014, IZA Discussion Paper 8018. Download · Source
- Who was studied: Applicants to nine Finnish economics and business programmes, whose entrance exams used negative marking.
- How: Observational. Each applicant’s number of skipped questions was compared with the number that would have maximised their chance of entry.
- What it found: Women skipped more questions and fell further short of the best number of answers, by answering too few.
- Why it matters for us: The same skipping pattern appears in a real high-stakes exam, not only in the lab.
- Where we use it: Learning science literature review, negative marking and the design rules.