The Trap Inside the Instant Answer

Give ChatGPT a math problem: it solves it. Give it an email: it writes it. Give it a broken agent: it finds the bug. The interaction feels successful because the task is finished.

But if the goal is learning, “Did the user get the answer?” is a surprisingly bad metric.

What we care about is whether the learner can take the underlying idea and apply it to a new problem, a different context, or a task they have not seen before.

We now have enough real-world evidence from researchers in our community to see this pretty clearly.

The second question cannot be answered from one conversation. It needs a task the learner has not seen, and a record of what they did on the last one.

Giving someone GPT can improve performance without improving learning

Bastani et al. [1] tested this with nearly 1,000 high-school mathematics students.

Students were assigned to a control condition without AI, GPT Base, or GPT Tutor. GPT Base was close to a normal GPT-4 chat interface. GPT Tutor also used GPT-4, but added teacher-written hints and instructions to guide students through the problem.

During practice, students using GPT Base performed 48% better than the control group. Students using GPT Tutor performed 127% better.

Then the AI was removed for an assessment.

Students who had used GPT Base performed 17% worse than the control group. Students who had used GPT Tutor did not show the same drop.

The bars compare groups with their classmates; they do not mean any student’s own scores fell. The ≈0 bar is drawn at zero because the GPT Tutor group’s assessment scores were statistically indistinguishable from classmates who never used AI.

That is the main result: the version of GPT that made practice easier also left students worse off on the later assessment. Adding tutoring guardrails largely removed that effect.

Both groups had access to GPT-4; the difference was how the system guided them through the problem.

The conversation logs also showed that students using GPT Base were more likely to ask for or copy complete solutions. GPT Tutor pushed students through more intermediate steps.

Therefore, a correct answer during practice is not enough to tell us whether someone learned. In some cases, LLMs might not be helpful for learning gains at all.

Bastani et al., PNAS 2025

LearnLM: did the learner generalize?

A recent LearnLM study gives us a more useful way to measure this.

Some of our friends at Google and Eedi studied 165 secondary-school students. After getting a math problem wrong, students received either a static hint, help from a human tutor, or tutoring generated by LearnLM and reviewed by a human tutor.

On a second attempt at the same problem:

The researchers then looked at how students performed on the first problem in the next unit:

Both panels use the same three conditions and the same 165 students; the second one is the measurement most tutoring products never take.

The main result here is different: LearnLM was very effective at helping students correct the immediate mistake, and students also performed well when they moved on to a new problem.

The study is small, and the difference between LearnLM and human tutoring is not statistically conclusive. The interesting part is the measurement itself.

The researchers did not stop at “Did the student get the original question right after receiving help?” They also looked at what happened on a new problem. That is much closer to what we care about in a learning system.

LearnLM + Eedi study (arXiv)

The learner has to do something first

This changes the product design. Suppose someone is learning to debug an agent, and the agent selects the wrong tool.

The fastest response is:

That fixes the agent, but it tells us almost nothing about what the learner understands. Instead, the system could first ask:

If the learner chooses output schema, we have learned something useful. They may not yet understand which part of the system controls tool selection.

Scarlatos and colleagues make the same argument on the model side: train tutor responses against the learner’s subsequent response rather than the immediate quality of the tutor’s own turn.

Now the system can address that specific gap and give them another agent where the same concept appears differently. The sequence becomes:

task → attempt → observe → support → new task

Without the attempt, there is very little to adapt to.

Scarlatos et al., AIED 2025

We also need to remember how the learner got the answer

This is where knowledge tracing becomes useful. Consider two learners who both finish a question correctly.

If the system records both as completed = true, it has lost most of the useful information.

Knowledge tracing is the study of estimating what a learner knows from the sequence of their interactions, rather than from the final state of each one.

Educational Data Mining has studied this problem for years through knowledge tracing: estimating what a learner knows from their sequence of interactions.

Work from our collaborators at EPFL tested whether LLMs could perform knowledge tracing directly. Fine-tuned LLMs performed roughly on par with standard Bayesian Knowledge Tracing approaches, although stronger specialized models still did better.

The main takeaway is simple: how someone reaches the answer matters for estimating what they know.

Neshaei et al., EDM 2024 (arXiv)

Khanmigo: having a tutor available is not enough

Khanmigo shows another problem. A two-year randomized study followed students across 18 Tennessee middle schools. Khanmigo was designed to coach students rather than simply provide answers.

Students assigned to Khanmigo improved by roughly 0.06–0.08 standard deviations over a school year.

Two years of access, 18 middle schools, and an effect of 0.06 to 0.08 standard deviations over a school year.

But students did not use the tutor very often. Although 96% tried Khanmigo at least once, the median student used it on only around one-third of practice days. More importantly, students used it in only 17% of exercise sessions where they made a mistake.

The main result for us is that a well-designed tutor does not help much if it is disconnected from the moments where the learner actually needs support.

A mistake is one of the strongest signals the system gets. If the learner simply moves on, that signal never changes how the learning should adapt next.

Oreopoulos & Low, NBER 2026

What we are building toward

This is the learning loop we care about at Scholé.

The LLM can generate explanations, examples, questions, simulations, and feedback. The surrounding system keeps track of what the learner tried, how they responded, how much help they needed, and what would be useful to test next.

The return leg is the load-bearing part. Repeating the same question tests recall; changing the surface tests transfer.

If someone learns that an ambiguous tool description can cause an agent to choose the wrong tool, the next test should not simply repeat the same question.

Give them a different agent. Change the tools. Change the symptoms. See whether they still recognize what is happening.

The same applies to sales, food handling, aviation, coding, governance, or learning how to use AI tools.

The result we care about is not whether the learner got through the exercise, but whether they can use the idea somewhere new.

• Learning science
• Knowledge tracing
• Transfer
• Adaptive learning
• AI proficiency