Step-based tutoring and problem decomposition

Lead. Breaking a problem into steps helps when each step the student writes is checked as it is written, and when hints make sure the student finishes the problem having done most of the reasoning. In VanLehn’s review, step-based tutors beat no tutoring by d = 0.76, against 0.31 for systems that only mark the final answer. Going finer than a step, into dialogue about each inference, added little: substep-based tutors beat step-based ones by only 0.16 (VanLehn 2011). So the right grain is one line of written working, usually one principle applied. Fading should run from a worked example, through the same example with the last steps left blank, to a whole problem, with a quick “what is your first step?” probe deciding where each student starts (Renkl, Atkinson and Maier 2000; Kalyuga 2007). The gains concentrate in novices. They shrink on answer-only exams like JEE: the Andes physics tutor gave 0.61 on show-your-work hour exams but 0.25 on the multiple-choice final, and nothing detectable for engineering and science majors (VanLehn et al. 2005).

Scope note. Every row below was opened and read in a downloaded copy on 2026-10-30, and every number was checked against the text. Almost all of this work is with US undergraduates or US school students in lab sessions or single courses. None is with Indian students, none is on JEE, and none is with first-generation or tribal students in a residential school. Four primary works the task named could not be obtained by an open route; they appear only under “Leads, not opened”.

Evidence table

Study	Design and population	n	Effect (unit)	What it means for the coach
VanLehn 2011	Review of experiments that compared two kinds of tutoring on the same content. STEM tasks with right and wrong answers. Mostly US college students, many short lab studies	99 pairwise comparisons in Table 1 outside the answer-based row; 28 step-based vs none, 26 substep vs none, 10 human vs none, 11 substep vs step	vs no tutoring (d): answer-based 0.31 (from Kulik and Kulik 1991, 165 studies), step-based 0.76, substep-based 0.40, human 0.79. Substep vs step 0.16; human vs step 0.21; step vs answer-based 0.40 (only 2 comparisons)	Feedback on each written step is the active ingredient. Finer dialogue does not add much. The plateau starts at the step
VanLehn et al. 2005 (Andes)	Quasi-experiment, US Naval Academy intro physics, five fall semesters 1999-2003. Andes sections vs control sections doing similar homework on paper. Same textbook and lectures; Andes only replaced homework	628 Andes and 438 control students on hour exams over 5 years; 931 students on the 2003 final	Hour exams (show work): d = 0.21 to 0.92 by year; pooled 2000-2003 d = 0.61. 2003 final (multiple choice, answers only): 0.25 overall; engineering majors 0.22 and science majors 0.18 (both not significant); other majors 0.5	Step feedback improved the practices the tutor required (drawings 1.21, variable definitions 0.69 average) but not the equations (0.11) or the answer subscore (-0.08). Gains on answer-only tests are much smaller
Ma, Adesope, Nesbit and Liu 2014	Meta-analysis of ITS studies, all ages and subjects	107 effect sizes, 14,321 participants	Random effects g: vs large-group teaching 0.42, vs textbook 0.35, vs human tutoring -0.11 (no difference). High school 0.40 (k = 14); physics 0.38 (k = 24); chemistry 0.16 (k = 2)	A realistic ITS effect for secondary physics is about 0.4 SD, not 0.76. The meta-analysis did not code step vs answer granularity
Steenbergen-Hu and Cooper 2013	Meta-analysis of ITS in K-12 mathematics, 1997-2010, mostly vs regular classroom teaching; many Cognitive Tutor field trials	26 reports, 34 samples	Mean g from 0.01 to 0.09 depending on the model. Larger for interventions shorter than a school year. Low achievers vs general students, fixed-effect model: on unadjusted effect sizes (Table 2) general 0.09 (k = 24) vs low achievers -0.42 (k = 2; random effects -0.23, not significant); on adjusted effect sizes (Table 3) general 0.04 (k = 14) vs low achievers -0.18 (k = 3; random effects -0.16, not significant). The text says three studies reported results for low achievers	In real school use the step-based math tutors averaged close to zero. Low achievers did worse. Supervision and fit matter more than the tutor
Kulik and Fletcher 2012 (IDA report, precursor of the 2016 RER paper)	Meta-analysis of ITS system evaluations, mostly US school, college and military training	45 studies	Mean 0.60 SD, median 0.63; tutored students did better in 41 of 45 studies. Programmes that attended to implementation did significantly better	Large effects are real in controlled use. Implementation decides whether they survive
Koedinger, Corbett and Perfetti 2012 (KLI)	Theoretical framework from the Pittsburgh Science of Learning Center, built on many classroom experiments	Not an effect study	Defines a knowledge component (KC) as a unit of knowledge inferred from performance on related tasks, typically a step of about 10 seconds. Separates memory and fluency, induction and refinement, and understanding and sense-making. Names integrative KCs: combining two steps is its own piece of knowledge	Tag each step of a lesson with the KC it practises. A ladder of rungs does not teach the step that joins the rungs; that join must be practised too
Koedinger et al. 2023 (PNAS)	Learning-curve models fitted to logs from tutors, games and online courses, late elementary to college, math, science and language	27 datasets, 6,946 learners, 1.3 million observations	Initial accuracy about 55% (lower half) vs 75% (upper half); learning rate about 0.1 log odds, or 2.5% accuracy, per practice opportunity, similar across students; about seven opportunities per KC to reach 80%	One lesson is one or two opportunities. A KC needs about seven, spread out. Weaker students start lower, not slower
Catrambone 1998	Four lab experiments, US college students learning Poisson probability from worked examples with or without a label naming a group of steps	150 (Exp 1), 180 (Exp 2), 90 (Exp 4)	Exp 2, novel problems solved out of 4 (n = 30 per cell): no label 2.33 now, 1.77 after a week; surface label 3.47 now, 1.80 after a week; abstract label 3.17 now, 3.00 after a week	Naming the purpose of each group of steps (a subgoal) improves transfer to changed problems. Abstract labels hold up after a week. Weaker students need labels tied to the problem’s features
Margulieux, Morrison, Guzdial and Catrambone 2016 (ICLS)	Lab RCT, US intro programming students, novices only (pre-test above 2 of 5 excluded). Worked example and practice pairs: no labels, given subgoal labels, or student writes own labels after training	120 analysed of 175	Given labels +7.8 and own labels +8.6 points out of 44 vs no labels; partial omega squared .08	Given and self-written subgoal labels worked equally well on average. Writing your own needs training first
Renkl, Atkinson and Maier 2000 (Freiburg report 140)	Exp 1: quasi-experiment, two Grade 9 classes in a German Hauptschule (lowest track), one physics lesson on electricity, backward fading vs example-problem pairs. Exp 2: lab RCT, US psychology undergraduates, probability, forward fading	Exp 1: 20 + 15; Exp 2: 54	Exp 1 near transfer 79.4% vs 62.2% (significant), far transfer 36.3% vs 21.1% (not significant). Exp 2 near transfer 53.9% vs 43.8% (significant), far transfer 38.7% vs 43.4% (not significant). Fading reduced errors during learning	Fading worked steps helps near transfer. It did not show far transfer. The one school study is the closest population here: low-track adolescents in physics
Kalyuga 2007	Review of expertise-reversal experiments, school and adult learners, including adaptive tutorials for high-school linear equations and kinematics	Many studies	Worked examples help novices and lose value or reverse for more knowledgeable learners. Adaptive tutorials for linear equations that used a rapid first-step test to choose how much was worked out beat non-adapted versions: knowledge gain effect sizes 0.46 (Kalyuga and Sweller 2004) and 0.55 (2005)	Route each student by a first-step probe. Fade worked steps one at a time as the student shows the step
Chi, Bassok, Lewis, Reimann and Glaser 1989	Think-aloud study of students studying worked mechanics examples (Halliday and Resnick), no college physics	10 recruited, 8 analysed (4 Good, 4 Poor)	Good students solved 82% of problems, Poor 46%. Good students produced 15.3 physics explanations per example vs 2.8	Correlational and tiny. It shows what students do with steps: strong ones explain why each step is there
Bisra, Liu, Nesbit, Salimi and Winne 2018	Meta-analysis of experiments that prompted learners to self-explain while studying or solving	64 reports, 69 effect sizes, 5,917 participants	Overall g = 0.55. Vs no extra explanation 0.67; vs instructor explanation 0.35. Studying worked examples 0.36; solving problems 0.48. High school 0.43. Content-general prompts 0.68 vs specific 0.51 (difference not significant). Multiple-choice prompts 0.24 (not significant, k = 2)	A short “why does this step follow?” prompt at key steps is worth adding. It beats giving the explanation, but much of its benefit is also available from a good explanation
Costa, Miranda and Melo 2022	Meta-analysis of whole-task programmes built with the 4C/ID model	12 studies, samples 22 to 160	d = 0.79 (95% CI 0.50 to 1.08); heterogeneity I squared 78.6%; grade level a significant moderator, larger in higher education	Weak support for whole tasks with fading support. Too few and too mixed studies to size an effect
Docktor and Mestre 2014	Synthesis of physics education research for the US National Research Council	Hundreds of studies	Explicit problem-solving frameworks, modelled and reinforced through a course, give higher course performance. Context-rich problems help when used with a strategy and groups. Self-diagnosis of errors is hard: one third to one half of students recognised using the wrong principle. Web homework vs paper: mixed results	Teach one fixed sequence for every problem (describe, plan, execute, check). Do not rely on students finding their own errors from a solution
Singh, Maries, Heller and Heller 2023 (arXiv book chapter)	Narrative review of physics problem-solving instruction by PER researchers, including the Minnesota cooperative-group work	Not an effect study	Four-step framework; outlining before solving; fading of scaffolds (cognitive apprenticeship). Summarises Yerushalmi et al. 2012: students allowed to regain 50% of lost points by correcting a test did better with their own resources than when given a detailed solution	Too much support during error correction can make reflection shallow. Give the student an incentive and little scaffolding when reviewing a test
Pardos and Bhandari 2024 (PLOS ONE)	Online RCT, US Mechanical Turk adults, algebra and statistics problems in an open-source tutor, ChatGPT hints vs human-authored hints vs no help	274	Gains: ChatGPT hints 17% vs no help 1.85% (significant); human hints not significantly different from ChatGPT. ChatGPT help failed quality checks on 32% of problems, cut to near 0% for algebra and 13% for statistics with self-consistency	AI-written step hints can teach, but a third failed checks before filtering. Every AI-written step needs automatic checks and an expert’s review before a student sees it

What the evidence says

What a “step” is

VanLehn (2011) defines a step-based tutor as one that lets students enter the steps they would write when showing all their work, and gives feedback and hints on each. A substep is finer: an inference students make in their head but do not write, such as why two events are independent. Every substep-based tutor built so far used natural-language dialogue (VanLehn 2011). Andes, the physics example, treats one equation, one vector drawn or one variable defined as a step. Its hints come in a fixed order: a pointing hint, a teaching hint, then a bottom-out hint that says exactly what to enter (VanLehn et al. 2005). KLI gives the cognitive version: a knowledge component is usually a unit task of about ten seconds, the application of one operator such as one theorem (Koedinger, Corbett and Perfetti 2012).

How fine the steps should be

VanLehn expected effectiveness to keep rising as the grain got finer. It did not. Effects rose from 0.31 (answer-based) to 0.76 (step-based) and then flattened: substep tutors 0.40 against no tutoring and 0.16 against step tutors, human tutors 0.21 against step tutors (VanLehn 2011). In the Why2 physics experiments he reports, human tutors, two dialogue tutors and a plain step-based text tutor were not reliably different, and all beat reading a textbook by about d = 1.0. The one exception was students without college physics given material written for students with it: there the human tutor won, because the material was above the students’ level (VanLehn 2011).

His explanation is the useful part. Step feedback narrows where the error is, so students can repair it. And hints right down to bottom-out keep students working until they finish the problem doing most of the reasoning, instead of quitting. Each finished step is a chance to strengthen or fix a knowledge component; a student who quits misses them (VanLehn 2011). So the step is fine enough, as long as each step needs only a little reasoning and the student is kept going to the end.

What step feedback improves, and what it does not

Andes improved exactly what it made students do. On show-your-work hour exams, Andes students drew vectors and defined variables far better (average effect 1.21 and 0.69) but wrote equations only slightly better (0.11) and got the final answer no more often (-0.08) (VanLehn et al. 2005). On the answer-only final, the overall effect was 0.25, and it was not significant for engineering or science majors (VanLehn et al. 2005). The authors suggest strong students learn as well from paper, and engineers had a parallel statics course. For JEE, which scores only the answer, this is the warning: expect a gain well below the headline.

Meta-analyses put ITS against ordinary teaching at about 0.4 SD for high school and for physics (Ma et al. 2014), and at about 0.6 SD in 45 controlled evaluations (Kulik and Fletcher 2012). In K-12 mathematics field use the average was near zero. For low achievers, the three studies that reported them gave negative averages: g = -0.42 (2 effect sizes, unadjusted) or -0.18 (3, adjusted) under a fixed-effect model, neither significant under random effects (Steenbergen-Hu and Cooper 2013).

Subgoals: grouping steps under a named purpose

Students who study worked solutions tend to memorise a linear list of steps tied to surface features, and fail when a new problem needs the steps changed (Catrambone 1998). A label naming what a group of steps achieves prompts students to explain its purpose to themselves and form a subgoal. Labels improved transfer to changed problems; abstract labels held up a week later when surface labels did not; students with weaker maths needed labels tied to the problem’s features (Catrambone 1998). In programming, given labels and labels students wrote themselves after training both beat no labels by about 8 points out of 44 (Margulieux et al. 2016).

Fading: from worked example to whole problem

Fading turns a worked example into a problem one step at a time. In backward fading, the last step is left blank first, then the last two, and so on. In a Grade 9 physics lesson at a low-track German school, backward fading beat example-problem pairs on near transfer (79% vs 62%); in a lab study with undergraduates, forward fading also helped near transfer (54% vs 44%). Neither showed a far-transfer gain, and fading reduced errors during learning (Renkl, Atkinson and Maier 2000). Kalyuga (2007) reports adaptive tutorials for high-school linear equations that used a quick first-step test to decide how much of the solution to show, and faded steps as students passed: they beat fixed versions with effect sizes of 0.46 and 0.55 on knowledge gain.

Self-explanation at the steps

In Chi et al. (1989), the students who learned most from worked mechanics examples explained why each line followed (15.3 physics explanations per example vs 2.8). That was a study of eight students, so it shows a pattern, not a cause. The experimental evidence is Bisra et al. (2018): prompting self-explanation gave g = 0.55 overall, 0.43 in high school, 0.36 while studying worked examples. It beat giving the same explanation (0.35), but a large share of its benefit was also available from a given explanation. Open prompts did at least as well as content-specific ones; multiple-choice prompts did not work (Bisra et al. 2018).

Whole tasks and physics practice

The 4C/ID model sequences whole tasks from simple to complex, fades support within each level, and adds part-task practice for routines. A meta-analysis of 12 studies found d = 0.79, but with heavy heterogeneity and larger effects in higher education (Costa, Miranda and Melo 2022). Physics education research supports an explicit framework, modelled and repeated through the course (Docktor and Mestre 2014), with scaffolds faded and students asked to outline before solving (Singh et al. 2023). Students struggle to diagnose their own errors from a solution sheet (Docktor and Mestre 2014). In one line of studies, too much support during test correction made reflection shallow (Yerushalmi et al. 2012, as summarised in Singh et al. 2023).

How much practice one lesson buys

Across 27 datasets, students needed about seven practice opportunities per knowledge component to reach 80%, gaining about 2.5 percentage points per opportunity. Weaker students started lower (55% vs 75%) but learned at a similar rate per opportunity (Koedinger et al. 2023). A single lesson after a wrong answer therefore cannot produce mastery. It starts a sequence.

AI writing the steps

ChatGPT-written hints for algebra and statistics produced learning gains (17% vs 1.85% with no help) and were not significantly different from human-written hints. But 32% failed quality checks before filtering; asking the model several times and keeping the modal answer cut this to near 0% for algebra and 13% for statistics (Pardos and Bhandari 2024). The learners were US adults on Mechanical Turk, and the problems were far easier than JEE.

Where it is weak or contested

1. VanLehn’s 0.76 is thin. It rests on 28 comparisons, many short lab studies with US college students. The answer-based 0.31 comes from an older review of different studies, and only two experiments compared step and answer feedback directly (0.40) (VanLehn 2011).
2. Andes is quasi-experimental. Sections were not randomised, the Andes instructors were co-authors, and the final-exam comparison needed a GPA adjustment because Andes sections had more engineers (VanLehn et al. 2005).
3. Field effects are smaller. Secondary ITS average about 0.4 SD (Ma et al. 2014), and K-12 maths ITS close to zero (Steenbergen-Hu and Cooper 2013).
4. Who benefits is contested. Andes found little aptitude-treatment interaction on hour exams, but on the final only the non-science majors gained (VanLehn et al. 2005). Low achievers did worse with K-12 maths tutors in a fixed-effect model, on two or three effect sizes, and the gap was not significant under random effects (Steenbergen-Hu and Cooper 2013). Worked steps help novices and can hurt knowledgeable students (Kalyuga 2007).
5. Fading shows near transfer only, in two small studies (Renkl, Atkinson and Maier 2000). The fuller journal studies were not obtained.
6. Subgoal evidence is mostly lab work with US undergraduates on probability and programming (Catrambone 1998; Margulieux et al. 2016). The semester-long field study was not obtained.
7. 4C/ID’s d = 0.79 comes from 12 mixed studies with high heterogeneity (Costa, Miranda and Melo 2022). Treat it as weak.
8. The learning-rate regularity (Koedinger et al. 2023) has been questioned in a 2026 preprint that I saw only in search results and did not open.
9. Population distance. No study here is from India, from JEE preparation, or from residential schools for tribal students. The nearest is one Grade 9 low-track physics class in Germany (Renkl, Atkinson and Maier 2000).

Design rules for an Avanti lesson

1. One step = one line of written working, usually one principle applied. Do not split it into dialogue substeps by default; finer grain added 0.16 over step-based tutoring (VanLehn 2011). This is consistent with dropping free-form Socratic dialogue.
2. Check each step when it is entered and flag the first wrong one. Then give hints in a fixed order: point to the error, teach the principle, then give the exact step (VanLehn et al. 2005; VanLehn 2011).
3. Make sure the student finishes the problem. The bottom-out hint exists so the student completes the reasoning instead of quitting; finishing is where the learning opportunities are (VanLehn 2011).
4. Name each rung as a subgoal. For the electric-field ladder: “find the charge enclosed”, “apply Gauss’s law”, “add up thin shells”. For weaker students tie the label to the problem; for stronger students use the abstract principle (Catrambone 1998; Margulieux et al. 2016).
5. Put conceptual steps in the step list, not only algebra. Andes improved only the practices it required: drawings and variable definitions, not answers (VanLehn et al. 2005). If mastery must be conceptual and procedural, both kinds must be steps that are checked. Rung 1 of the ladder (“flux depends only on enclosed charge”) is a checked step, not a preamble.
6. Fade backward. Show the rung fully worked, then the same structure with the last step blank, then more blanks, then the whole problem (Renkl, Atkinson and Maier 2000).
7. Start each student where their first step says. Ask “what is your first step?” on the target. A correct first step skips the worked rungs; a wrong one starts at the rung that step belongs to (Kalyuga 2007).
8. Ask one open “why” at the key steps, typed in the student’s words, not multiple choice (Bisra et al. 2018). Do not attach an explanation-on-demand button to the same step (Schworm and Renkl 2006, as summarised in Bisra et al. 2018).
9. End with the joining step, unassisted. The rungs do not teach the combination; the combination is its own knowledge component (Koedinger, Corbett and Perfetti 2012). The lesson closes with the original target and a new isomorph, both with no hints. Only those count towards mastery.
10. Schedule the follow-ups. About seven opportunities per knowledge component are needed (Koedinger et al. 2023). One lesson is not mastery; the coach queues isomorphs in later sessions.
11. Measure on answer-only, unassisted items, and report conceptual and procedural accuracy separately. Gains on answer-only exams were less than half those on show-your-work exams (VanLehn et al. 2005), and a system’s own tests overstate gains (Kulik and Fletcher 2012).
12. Expect most gain in weaker students, and keep an adult on the slot for them. Strong students may gain little from worked steps (Kalyuga 2007; VanLehn et al. 2005), and low achievers did worse with unsupervised maths tutors (Steenbergen-Hu and Cooper 2013).
13. Every AI-written step is checked automatically and by an expert before use. A third of ChatGPT’s hints failed checks before filtering (Pardos and Bhandari 2024). This matches the Purna concept’s expert verification.
14. After tests, review with little scaffolding and a reason to do it. Students given a full solution reflected more shallowly than students using their own notes (Yerushalmi et al. 2012, as summarised in Singh et al. 2023).

References

All files are archived in the session scratchpad at sources/step_tutoring/, with SHA-256 checksums in SHA256SUMS.txt.

• Bisra, K., Liu, Q., Nesbit, J. C., Salimi, F., and Winne, P. H. (2018). Inducing self-explanation: a meta-analysis. Educational Psychology Review, 30, 703-725. https://doi.org/10.1007/s10648-018-9434-x (copy at https://gwern.net/doc/psychology/spaced-repetition/2018-bisra.pdf). Opened 2026-10-30. File: bisra2018_selfexpl_meta.pdf.
• Catrambone, R. (1998). The subgoal learning model: Creating better examples so that students can solve novel problems. Journal of Experimental Psychology: General, 127(4), 355-376. https://doi.org/10.1037/0096-3445.127.4.355 (copy hosted on mrbartonmaths.com). Opened 2026-10-30. File: catrambone1998_subgoal.pdf.
• Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., and Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182. https://doi.org/10.1207/s15516709cog1302_1 (scan hosted by ASU Learning Champions Lab, OCR made locally). Opened 2026-10-30. File: chi1989_selfexpl.pdf.
• Costa, J. M., Miranda, G., and Melo, M. (2022). The four-component instructional design model (4C/ID): A meta-analysis on use and effect. Learning Environments Research, 25(2), 445-463. https://doi.org/10.1007/s10984-021-09373-y (accepted version, ISCTE-IUL repository). Opened 2026-10-30. File: costa2022_4cid_meta.pdf.
• Docktor, J. L., and Mestre, J. P. (2014). Synthesis of discipline-based education research in physics. Physical Review Special Topics - Physics Education Research, 10, 020119. https://doi.org/10.1103/PhysRevSTPER.10.020119. Opened 2026-10-30. File: docktor2014_per_synthesis.pdf.
• Kalyuga, S. (2007). Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review, 19, 509-539. https://doi.org/10.1007/s10648-007-9054-3. Opened 2026-10-30 (re-read from this plan’s archive). File: kalyuga2007_reversal_review.pdf.
• Koedinger, K. R., Corbett, A. T., and Perfetti, C. (2012). The Knowledge-Learning-Instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36(5), 757-798. https://doi.org/10.1111/j.1551-6709.2012.01245.x (authors’ final version, CMU KiltHub). Opened 2026-10-30. File: koedinger2012_kli.pdf.
• Koedinger, K. R., Carvalho, P. F., Liu, R., and McLaughlin, E. A. (2023). An astonishing regularity in student learning rate. PNAS, 120(13), e2221311120. https://doi.org/10.1073/pnas.2221311120 (full text from Europe PMC, PMC10068755). Opened 2026-10-30. File: koedinger2023_learning_rate.xml.
• Kulik, J. A., and Fletcher, J. D. (2012). Effectiveness of intelligent tutoring systems. IDA Document D-4664, Institute for Defense Analyses. Opened 2026-10-30 (re-read from this plan’s archive). File: KF2016_its_meta.pdf.
• Ma, W., Adesope, O. O., Nesbit, J. C., and Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901-918. https://doi.org/10.1037/a0037123. Opened 2026-10-30 (re-read from this plan’s archive). File: MANL2014_its_meta.pdf.
• Margulieux, L. E., Morrison, B. B., Guzdial, M., and Catrambone, R. (2016). Training learners to self-explain: Designing instructions and examples to improve problem solving. Proceedings of ICLS 2016, Vol. 1. https://hdl.handle.net/20.500.14694/10111 (Georgia State ScholarWorks). Opened 2026-10-30. File: margulieux2016_selfexplain_training.pdf.
• Pardos, Z. A., and Bhandari, S. (2024). ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills. PLOS ONE, 19(5), e0304013. https://doi.org/10.1371/journal.pone.0304013. Opened 2026-10-30 (from this plan’s ai_tutors archive). File: pardos2024_chatgpt_hints.pdf.
• Renkl, A., Atkinson, R. K., and Maier, U. H. (2000). From example study to problem solving: Smooth transitions help learning. Research Report No. 140, Institute of Psychology, University of Freiburg. https://www.psycharchives.org/handle/20.500.11780/196. Opened 2026-10-30. File: renkl2000_smooth_transitions_report.pdf. (Later published as Renkl, Atkinson, Maier and Staley 2002, Journal of Experimental Education; that version was not opened.)
• Singh, C., Maries, A., Heller, K., and Heller, P. (2023). Instructional strategies that foster effective problem-solving. arXiv:2304.05585 (book chapter). https://arxiv.org/abs/2304.05585. Opened 2026-10-30. File: maries2023_ps_strategies_arxiv.pdf.
• Steenbergen-Hu, S., and Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K-12 students’ mathematical learning. Journal of Educational Psychology, 105(4), 970-987. https://doi.org/10.1037/a0032447 (DukeSpace copy). Opened 2026-10-30. File: steenbergenhu2013_its_k12math.pdf.
• VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197-221. https://doi.org/10.1080/00461520.2011.611369. Opened 2026-10-30 (re-read in full from this plan’s archive). File: V2011_tutoring.pdf.
• VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., and Wintersgill, M. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15(3), 147-204. Author copy via Wayback Machine of https://www.public.asu.edu/~kvanlehn/distrib/AndesLessonsLearnedForWeb.pdf. Opened 2026-10-30. File: vanlehn2005_andes.pdf.

Leads, not opened

Routes tried for each: Avanti’s GCS archive, this plan’s archive, author and institutional repositories, ERIC, Wayback, publisher open access. The sci-hub fallback was refused by this session’s permission system and was not retried.

• Kulik, J. A., and Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42-78. Paywalled; the 2012 IDA report was used instead.
• Steenbergen-Hu, S., and Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331-347. The DukeSpace link found returned the 2013 paper; no copy of the 2014 one was found.
• Margulieux, L. E., and Catrambone, R. (2016). Improving problem solving with subgoal labels in expository text and worked examples. Learning and Instruction, 42, 58-71. The GSU repository returned a different paper (the ICLS paper above).
• Margulieux, L. E., Morrison, B. B., and Decker, A. (2020). Reducing withdrawal and failure rates in introductory programming with subgoal labeled worked examples. International Journal of STEM Education, 7, 19. Open access, but the publisher blocked automated download and the Wayback copy was truncated. This is the semester-long field test of subgoal labels and is worth opening by hand.
• Atkinson, R. K., Renkl, A., and Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Effects of self-explanation prompts and fading worked-out steps. Journal of Educational Psychology, 95(4), 774-783. No open copy found.
• Renkl, A., and Atkinson, R. K. (2003). Structuring the transition from example study to problem solving in cognitive skill acquisition. Educational Psychologist, 38(1), 15-22. No open copy found.
• Sweller, J., and Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2(1), 59-89. No open copy found.
• Chi, M. T. H., de Leeuw, N., Chiu, M.-H., and LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477. Not obtained.
• Heller, P., Keith, R., and Anderson, S. (1992). Teaching problem solving through cooperative grouping. Part 1. American Journal of Physics, 60(7), 627-636; and Heller, P., and Hollabaugh, M. (1992). Part 2. American Journal of Physics, 60(7), 637-644. No open copies found; described here only through Docktor and Mestre (2014) and Singh et al. (2023).
• Barbieri, C. A., Miller-Cotto, D., Clerjuste, S. N., and Chawla, K. (2023). A meta-analysis of the worked examples effect on mathematics performance. Educational Psychology Review, 35, 11. Publisher PDF not retrieved; its reported finding that self-explanation prompts weakened the worked-example effect is not used here.
• Kalyuga, S., and Sweller, J. (2004, 2005) adaptive tutorial studies; Yerushalmi et al. (2012); Schworm and Renkl (2006); Heffernan and Koedinger (1997). Cited only as summarised in opened reviews.
• A 2026 arXiv preprint revisiting the learning-rate regularity (arXiv 2605.01690), seen only in search results.