Teaching According to Students' Aptitude: Personalized Mathematics Tutoring via Persona-, Memory-, and Forgetting-Aware LLMs

AI summary

Overview

Research area: Natural Language Processing applied to education — specifically LLM-based intelligent tutoring systems, personalized instruction, and knowledge tracing in mathematics learning.

Technical level: Advanced. The paper assumes familiarity with LLM agent pipelines, retrieval-augmented generation, embeddings and rerankers, and knowledge tracing models (BKT, DKT, AKT, LPKT, SimKT).

Scope in one sentence: The paper introduces TASA, a tutoring framework that combines a student persona, an event memory bank, and a cognitive forgetting curve so that an LLM tutor can adapt mathematics instruction to how much a student is likely to have retained, not just what they once answered correctly.

What This Paper Is About

Most LLM tutors respond with general-purpose, uniform instruction, and existing personalization methods condition responses on a student's past interactions as if that history were a static snapshot. The authors argue this ignores a well-established cognitive reality: learners forget, revisit, and reconstruct knowledge over time, and they differ in learning pace and retention — a problem that is especially acute in mathematics, where proficiency can degrade within weeks or months without reinforcement. The goal of TASA is to make the tutor's behavior track each student's current retention state by decaying retrieved persona and memory information before generating the next explanation and practice question.

Key Contributions

1. TASA framework: An LLM-driven mathematics tutoring framework that jointly models each student's persona, event memory, and forgetting dynamics to produce personalized instruction.
2. Learnable forgetting curve: Personalized mastery decay is formalized with a forgetting curve grounded in cognitive psychology and knowledge tracing, estimating concept-level temporal mastery decay and modulating both persona strengths and memory traces inside the tutoring loop.
3. Empirical validation: Mathematics tutoring studies across four public benchmarks and three LLM backbones showing TASA outperforms state-of-the-art LLM tutors on learning gains, learning efficiency, and personalization quality, with ablations isolating the effect of each module.

Main Findings

• Best or second-best across all settings: TASA consistently achieves the top or runner-up score on Normalized Learning Gain (Δ-NLG) and Response Personalization (Win Rate) across four benchmarks and three LLM backbones, which the authors present as evidence of generalization and robustness across architectures.
• Gains over the strongest baseline: Compared with TutorLLM, TASA improves the average Δ-NLG by 11.4% and the Win Rate by 19.7%.
• Largest benchmark-specific gains: On NIPS34 and Bridge2006, TASA achieves up to 9% higher learning gain, which the authors attribute to integrating time-dependent forgetting.
• Backbone-specific gains: On Llama3.1-8B-Instruct, TASA surpasses TutorLLM by 8.5% in Δ-NLG and 9.9% in personalization. Highest reported TASA numbers on that backbone are 59.4 (Assist2017), 54.9 (NIPS34), 62.3 (Algebra2005), and 53.9 (Bridge2006) for Δ-NLG, with Win Rates of 86.1, 82.5, 76.9, and 92.1 respectively.
• Every module matters: Removing the forgetting score drops Δ-NLG by an average of 6.8%; removing persona drops it by 11.5% on average and removing memory by 4.9% on average — the authors conclude all three components are essential.
• Multi-turn interaction is necessary, with diminishing returns: Δ-NLG rises steadily from 4 to 20 dialogue rounds and gradually saturates after that, motivating the default 20-round session design (10 student queries and 10 tutor responses).
• Retrieval weighting is not sensitive: TASA stays stable across λ in the range [0.3, 0.7] on all three backbones, with the best results at the default λ = 0.5.
• DKT is the best knowledge tracing instantiation: TASA_DKT reaches the highest average Δ-NLG (57.6), outperforming the simple history-based mastery approximation by 11.0% and other neural KT models by 9% on average. TASA_LPKT is the next strongest at 53.5, followed by TASA_AKT (50.7), TASA History (46.6), and TASA_SimKT (43.2).
• Qualitative judge results: GPT-5-as-judge evaluations indicate TASA's responses are more contextually aligned, pedagogically consistent, and adaptive to individual learner states.
• Reporting caveat: The Experimental Settings state that results are reported as the mean and standard deviation across 5 random seeds, while the Table 1 caption says the mean (std) is over three random runs. The paper's content contains this inconsistency and does not reconcile it.

Methodology in Plain English

TASA builds a profile of each student from their past interaction history using two LLM-based "generators":

1. Persona Generator reads the student's full history and writes structured entries describing mastery across knowledge concepts (for example, strong at basic arithmetic but weak at multi-step word problems), tagged with the relevant concept keywords.
2. Memory Generator reads recent interactions and writes timestamped event records (for example, a specific fraction-addition error on a particular question), also tagged with concepts.

Both sets of entries are encoded into vectors with a text encoder and stored in two banks — one for persona, one for memory. When a student asks something new, TASA scores entries using a hybrid similarity that blends semantic similarity of the description with concept-level keyword alignment, keeps the top-K from each bank, and then reranks with a cross-encoder to keep the top 3 from each.

The distinctive step is what happens next. A knowledge tracing model (DKT in the main configuration) estimates the student's current mastery probability for each concept. Combining that mastery estimate with how long it has been since the student last practiced the concept produces a forgetting score between 0 and 1 — higher means more likely to have forgotten. This is derived from an exponential memory-decay assumption and then replaced with a cheaper rational approximation that preserves the same behavior: the score rises with elapsed time and falls as mastery rises.

A "Rewriter" step then uses the LLM's zero-shot ability to revise the retrieved persona and memory descriptions in light of the forgetting score — turning a statement like "the student has mastered fractions" into a version reflecting how much of that mastery likely remains. A Generator finally produces the response given the dialogue session plus the forgetting-adjusted persona and memory: a short explanation of the student's latest answer and a next practice question calibrated to their current retention state. At the end of a session, the accumulated turns are merged back into the long-term history so future retrieval and forgetting estimates stay current.

Experimental setup: Four public benchmarks from knowledge tracing and intelligent tutoring research — ASSISTments2017, NIPS34, Algebra2005, and Bridge2006 — are used. Assist2017 has 1,708 students, 3,162 questions, 102 KCs and 942,816 interactions; NIPS34 has 4,918 students, 948 questions, 57 KCs and 1,382,727 interactions; Algebra2005 has 574 students, 210,710 questions, 112 KCs and 809,694 interactions; Bridge2006 has 1,138 students, 207,856 questions, 493 KCs and 3,679,199 interactions. The higher question counts in the latter two come from step-level decomposition of multi-step problems. Baselines are Vanilla-ICL, MathChat, TutorLLM, and PSS-MV, all given the same student history as TASA. GPT-4o-mini plays the student role; gpt-oss-120b, Qwen3-4B-Instruct, and Llama3.1-8B-Instruct serve as tutor backbones; GPT-5 acts as an impartial judge for personalization. Δ-NLG is computed as (ACC_Post − ACC_Pre) / (1 − ACC_Pre).

Why This Matters

Impact on research: The paper makes a case that temporal dynamics — forgetting and retention — belong inside the tutoring generation loop, not just inside a predictive knowledge tracing model. It bridges two research lines (LLM tutoring agents and knowledge tracing) by using KT estimates not merely to predict future performance but to actively rewrite the student context an LLM conditions on. It also provides an ablation-supported claim that a static-memory assumption is a measurable weakness, and offers a dataset- and backbone-comparable evaluation spanning four benchmarks.

Real-world applications:

• Adaptive math practice platforms that decide whether a struggling student needs remediation of previously learned material or can move on to harder problems.
• Spaced-review scheduling, where the forgetting score identifies which concepts are decaying and should be revisited soonest.
• Longitudinal tutoring over weeks or months, where a student's state changes substantially between sessions and a static profile would misrepresent them.
• Underserved-region education, the motivation the authors cite for AI tutoring generally — delivering individualized guidance where high-quality human tutoring is scarce.

Industry relevance: EdTech products, learning-management systems, and tutoring services that already log student-question interactions with timestamps and concept tags have exactly the data TASA requires. The framework is tuning-free at the rewriting step, uses standard knowledge tracing implementations, and showed stable behavior across λ values from 0.3 to 0.7, suggesting it could be integrated without delicate hyperparameter tuning — though the paper does not report deployment costs, latency, or production-scale evaluations.

Future Directions

1. Beyond mathematics: The authors explicitly plan to extend TASA to other educational domains and learner levels.
2. Multi-agent collaboration: Richer interactions through multiple collaborating agents are named as a next step.
3. Real-world deployment with human learners: Moving from simulated student agents (GPT-4o-mini) to actual students is stated future work; the paper does not report results with human learners.
4. Open questions the paper raises but does not resolve: How forgetting curves should be individualized per student rather than relying on a shared stability constant; how the framework behaves with knowledge tracing models other than DKT and LPKT in non-mathematics domains; and why reported results are described as over both five and three random runs in different parts of the paper.

Target Audience

Researchers and practitioners working on LLM-based educational agents, personalized learning systems, and knowledge tracing — particularly those building tutoring products that need to reason about what a student still remembers rather than only what they once got right. It is also useful for cognitive-science-informed AI researchers interested in embedding forgetting curves into LLM context construction. Readers without background in knowledge tracing or retrieval-augmented LLM pipelines will likely need to consult the cited KT literature first.

Authors’ abstract

Large Language Models (LLMs) are increasingly integrated into intelligent tutoring systems to provide human-like and adaptive instruction. However, most existing approaches fail to capture how students' knowledge evolves dynamically across their proficiencies, conceptual gaps, and forgetting patterns. This challenge is particularly acute in mathematics tutoring, where effective instruction requires fine-grained scaffolding precisely calibrated to each student's mastery level and cognitive retention. To address this issue, we propose TASA (Teaching According to Students' Aptitude), a student-aware tutoring framework that integrates persona, memory, and forgetting dynamics for personalized mathematics learning. Specifically, TASA maintains a structured student persona capturing proficiency profiles and an event memory recording prior learning interactions. By incorporating a continuous forgetting curve with knowledge tracing, TASA dynamically updates each student's mastery state and generates contextually appropriate, difficulty-calibrated questions and explanations. Empirical results demonstrate that TASA achieves superior learning outcomes and more adaptive tutoring behavior compared to representative baselines, underscoring the importance of modeling temporal forgetting and learner profiles in LLM-based tutoring systems.