Self-Explanation as a Feedback Strategy in SAT Math

This piece is about self-explanation (SE) as a feedback strategy in SAT Math, and the argument is simple: a wrong answer tells you almost nothing, but asking a student to explain their reasoning tells you exactly where it broke.

What self-explanation does in the brain during math problem-solving

Self-explanation is the act of generating an explanation for yourself, out loud or on paper, while you're still inside the problem, rather than reading a worked solution and nodding along. That generative step is what does the work, because it forces the brain to connect new material to what it already knows, catch its own contradictions, and patch gaps in its reasoning on the fly.

Wong, Lawson, and Keeves ran a study in 2002 that pinned this down with more precision than "explaining helps." They found SE had no direct effect on performance at all, with the effect fully mediated by knowledge access and knowledge generation, knowledge generation being the strongest predictor.

That finding reframes what SE is for. It isn't a rehearsal technique where repeating steps burns them into memory. A student can repeat a quadratic formula correctly without ever generating a new connection about why it works, and by this research, that repetition alone will not move performance. The connection-making is the mechanism. Everything else is scaffolding around it.

SE also isn't one fixed exercise. It runs along a continuum: at one end, open-ended prompts like "explain how you solved this," and at the other, menu-based formats where a student picks from a set of preset explanations. The form of the prompt changes which mental process gets activated, so "doing SE" can mean very different cognitive work depending on how the question is asked.

What a wrong answer hides about a student's thinking

A wrong answer tells a student that something failed. It does not tell them what. Standard SAT practice produces a binary result, right or wrong, and at best tags it with a category like "quadratics" or "systems of equations". That label points a student back to a topic, not to the specific step in their reasoning that collapsed.

The SAT Math section is built to reward transfer: taking a method you know and applying it to a problem shape you haven't seen before, under a countdown clock. A student who can execute a procedure without understanding why it works will hit a wall on a problem with unfamiliar dressing, no matter how many practice sets they've ground through.

Rittle-Johnson's 2024 review backs this up with a split finding that matters a great deal here. SE has strong evidence behind it for building conceptual knowledge, procedural knowledge, and transfer that happens immediately after instruction. But evidence for long-term procedural transfer, the kind a student needs weeks later on test day, is limited. In other words, the exact benefit SAT Math is demanding (durable transfer under pressure, long after the lesson) is the one SE struggles hardest to guarantee. That doesn't mean SE is unreliable; it means the design of the prompts and the feedback around them has to be deliberate, not an afterthought bolted onto a problem set.

The Digital SAT's adaptive structure adds a wrinkle. A student who guesses right on a hard question and never understood the concept behind it looks, to the algorithm, identical to a student who nailed it on solid footing. The adaptive engine optimizes for difficulty, not for understanding, so that gap stays invisible until it resurfaces on a later question the student can't guess their way through.

SE fills that exact hole. It produces something a score report and an adaptive algorithm cannot: a written trace of the reasoning itself, in the student's own words, that someone can actually read and evaluate.

Turning a practice problem into a diagnostic tool

Once a student writes down their reasoning instead of just circling an answer, that explanation becomes data. Read correctly, it maps exactly where solid understanding ends and guesswork begins.

A student who gets the right answer but can't explain why has procedural fluency sitting on top of nothing. That's a fragile setup that tends to collapse the moment the problem shifts shape, and SE is what exposes the fragility before it costs points on test day. Flip it around: a student who explains their reasoning well but lands on the wrong final answer is probably dealing with a careless slip, like an arithmetic error, not a conceptual gap. That student is lower risk than the one who got the right number for the wrong reasons.

Nakamoto and colleagues built an evaluative framework in 2025 that tests this directly. It combined rubric-based scoring of SE quality with handwritten response data from middle schoolers, and models trained on SE quality alone hit a substantial accuracy rate for detecting what the framework calls a "learning impasse," a point where a student struggles or fails to grasp a key concept. Accuracy climbed further once that SE data was combined with other operational data from the session. That's the same gap-detection problem SAT prep faces, just measured in a different classroom.

There's a classroom-tested version of this same idea in AP exam prep, because it's concrete. AP math educator Jean Adams, at Flamingo Math, runs a timed drill called a "Sprint": the free-response question appears on a screen, students write their answers in a printed booklet, and then they compare their own work against the official scoring guidelines. That structure does two things at once. It forces students to produce explanation-style reasoning under time pressure, and it benchmarks that reasoning against an expert standard immediately, while the thinking is still fresh. The same workflow maps directly onto SAT Math's extended-reasoning questions, or really any practice session where a worked solution already exists to compare against.

What gets surfaced depends on which type of prompt is used. Open-ended prompts tend to expose a student's own idiosyncratic reasoning patterns, the specific way their mind jumps from step to step. Menu-based prompts reveal something different: which conceptual frameworks a student reaches for, or fails to reach for, when given a set of options to choose from. Rittle-Johnson's 2024 review notes that menu-based SE tends to help students with weaker foundations more, while open-ended SE produces better long-term learning outcomes.

The real bottleneck: grading open-ended explanations at scale

All of this diagnostic value depends on one condition: someone has to read the explanation. In a typical SAT prep classroom or course, that condition almost never gets met.

Research on ASSISTments, a widely used online math learning platform, found that only a small share of open-ended responses ever get scored by a teacher, and an even smaller fraction get any feedback at all. That's the gap between what SE could reveal and what actually gets processed in a real classroom.

It would be easy to write this off as a problem for underfunded schools stretched too thin to keep up. It isn't. Even well-staffed programs hit the same wall once SE gets deployed at the frequency it actually needs to generate useful data: multiple problems per session and multiple sessions per week. No teacher, however well resourced, can read and meaningfully respond to that volume of open-ended writing across a full roster of students, every week, on top of everything else the job requires.

So SE quietly turns into a box-checking exercise. Students write something. Nothing gets read. No gap gets read or evaluated. The entire diagnostic value of the exercise evaporates, even though the student did the work that was supposed to generate it.

What AI grading of self-explanations can and cannot do

This is the obstacle that makes AI grading necessary rather than merely convenient. It only works under specific conditions, and those conditions have limits.

LLM-based grading works when the model gets an explicit rubric to grade against. It fails in predictable ways when that rubric is missing, or when a student expresses a correct idea using language the model wasn't trained to recognize. The research backs this distinction up cleanly. Studies that handed the model an explicit, theoretically grounded rubric consistently beat studies that relied on minimal prompting. In physics explanation grading specifically, LLMs given detailed rubrics can approach the agreement levels typically seen between human graders. That held most reliably at the extremes, meaning clearly strong or clearly weak responses, and agreement dropped off for the murkier, mid-level answers that are hardest for humans to grade too. Pairing an instructor's rubric with a model answer cut the gap between AI grades and instructor grades substantially.

The time savings involved were described as substantial in the comparisons that measured them, which is what would make routine SE grading operationally realistic for large SAT prep programs where teacher time is the scarce resource.

The failure mode is specific and documented, not speculative. Studies that gave the model only minimal context reported agreement scores as low as an F1-score of 0.48, and models broke down systematically when students expressed correct understanding in vague, individualized, or non-standard phrasing. That's a real risk in SAT prep specifically: students from non-dominant language backgrounds can write something semantically correct but syntactically unusual, and a grader calibrated on standard academic prose will mark it down for no good reason.

There's a second piece of this that points toward where the field is headed next: Nakamoto's earlier 2024 study looked at whether a standardized sample answer for an SE prompt could be generated automatically from a pool of unsupervised student responses, using vectorization, clustering, and extraction. Across more than a thousand SE answers tested, a strong majority of the quizzes produced sample answers that contained all the necessary knowledge components. That suggests the rubric itself is becoming something a system can build, not just something a teacher has to write from scratch.

None of this makes AI grading a solved problem. It makes rubric quality the variable that determines whether the grading is trustworthy, and that means building an SE system responsibly means investing in the rubric design, not just plugging in a model and hoping the output holds up.

The SE feedback loop and teacher use of score data

Layering SE data on top of a standard score report stops a teacher from getting a flat list of names next to missed topics. They get a map of why each student's reasoning failed on those specific topics, and that's a different kind of input for deciding what to teach next.

Standard SAT Suite score reports are built to help educators make data-driven decisions, tying skill and knowledge statements to specific SAT Content Domains. That's genuinely useful. It tells a teacher which domains are weak across a roster. What it cannot do is explain the reasoning error underneath a weak domain score, because the report was built to capture outcomes, not reasoning.

SE records fill precisely that hole. What a student says about their own thinking is the richest signal available in this kind of pipeline, because it captures whether a student reached for the right concept and misapplied it, or never identified the right concept in the first place, something item-response data cannot show.

Those two situations call for completely different responses. A tool that combines SE quality scores with performance data can distinguish a student who just needs more reps on a procedure from a student who needs a real conceptual intervention, a distinction that changes what a teacher actually does in the next session. The Nakamoto 2025 model, which combined SE quality features with handwriting and action data, is the research proof that this kind of layered diagnostic is possible. The practical version of that research is a dashboard that shows a teacher which students are stuck, on which concepts, and why.

AI-powered systems that handle content creation, assessment management, and detailed analytics together let tutors and teachers stop spending their time managing the sheer volume of grading and start spending it on what each student's reasoning actually reveals. That shift, from processing papers to interpreting thinking, is what the SE feedback loop makes possible once the grading bottleneck is handled by something other than a single overworked human.

Why SE quality is uneven

There's a real objection to all of this, and it deserves a straight answer instead of a dodge. Students with weaker prior knowledge and lower intrinsic motivation tend to write lower-quality explanations. The SE feedback loop is least reliable for exactly the students who need it most, unless the system is built to account for that from the start.

Rittle-Johnson's 2024 review doesn't paper over this. It says directly that children's self-explanations "often are not sophisticated or generalizable". And yet prompting students to generate them still tends to improve learning, even when the explanations themselves are rough or incomplete. So the act of attempting to explain carries value on its own, even when the explanation is weak. But that value only converts into something useful if the weak explanation actually gets evaluated. An unread explanation, however imperfect, never triggers the diagnostic function SE is supposed to serve in the first place.

A 2025 study in Interactive Learning Environments, looking at junior high students, found that intrinsic motivation was hypothesized to correlate positively with SE quality, and SE quality in turn predicted academic achievement. That creates a real dependency, and it's a live concern in test prep specifically, where motivation is often extrinsic and deadline-driven rather than curious and self-directed. Mandate SE in a context where students are grinding toward a score deadline, and some of them will produce surface compliance, writing something that sounds plausible without doing the actual thinking, and that surface compliance can fool peers and poorly calibrated AI graders alike.

The research points toward a design response rather than a reason to abandon SE altogether. Menu-based prompts lower the cognitive load for students with weaker foundations and offer scaffolding that raises the odds of getting a diagnostically useful response in the first place. Training students explicitly in how to self-explain, scaffolding the prompts themselves, and asking students to explain specifically why an incorrect answer is incorrect all give lower-performing students more structure to work within, rather than leaving them to generate elaborate reasoning from a blank page. Uneven SE quality is a constraint that tells you how the prompts need to be built for the students who need the most support, not a flaw that disqualifies the approach.