Simulating Students with Large Language Models: A Review of Architecture, Mechanisms, and Role Modelling in Education with Generative AI

AI summary

Overview

Research area: Generative AI in education, specifically the use of large language models (LLMs) to build "Simulated Students" — virtual learner agents used to test pedagogy, curriculum and assessment.

Technical level: Intermediate. The paper is a survey, so it is readable without deep engineering background, but it covers cognitive architectures, knowledge tracing models (DKT, DKVMN, AKT, LLM-KT) and multi-agent frameworks that assume some familiarity with machine learning terminology.

One-sentence scope: A thematic review of empirical and methodological studies that use LLMs to simulate student behaviour, synthesising architectural patterns, cognitive and affective modelling mechanisms, evaluation practices and open research gaps.

What This Paper Is About

Systematically evaluating teaching strategies, curricula and assessments on real learners is slow, costly and sometimes risky. Simulated Students — virtual agents that behave like learners — offer a low-risk, scalable alternative. This paper reviews how recent research builds such agents with LLMs: what cognitive architectures they use, how faithfully they mimic human learning behaviour, and whether they can actually improve teaching and instructional design. To the authors' knowledge, it is the first dedicated literature review focused specifically on the simulation of student roles and mechanisms using LLMs.

Key Contributions

1. A dedicated LLM-focused review of Simulated Students. The authors state this is the first literature review specifically on simulating student roles and mechanisms with LLMs, deliberately concentrated on recent publications because of how fast LLMs have evolved.

2. A unifying taxonomy and set of architectural patterns. The paper organises the field into foundational concepts, architectural patterns, cognitive/affective/trait modelling mechanisms, task generation and evaluation protocols, applications, limitations and open problems, spread across nine sections.

3. A three-way comparison of knowledge modelling strategies. Table 1 contrasts Direct Simulation via Prompts, Knowledge Tracing, and Knowledge Graphs and Heuristics, listing the specific advantages and limitations of each.

4. A structured, transparent review process. A PRISMA-based search across Scopus, Web of Science, IEEE Xplore, ACM Digital Library and Google Scholar, with the selection process carried out by two independent reviewers and cross-checked using the Gemini and ChatGPT LLMs.

Main Findings

• Pre-LLM simulated students were rigid and rarely validated. Early systems such as SCHOLAR (semantic nets to track known concepts) and Anderson's LISP Tutor (ACT-R, Model-Tracing against an Ideal Student Model) offered high fidelity and explainability but required costly knowledge engineering and could not anticipate every possible student error. A 2024 systematic review covering 2010–2019 literature found that Simulated Students models represented only "narrow aspects of student learning", and that almost half of simulated learner studies failed to provide any formal validation of their simulations.

• Bayesian Knowledge Tracing remains a foundation. Introduced by Corbett and Anderson in 1994, BKT uses a Hidden Markov Model with four core parameters — Prior, Learning (Transition), Guess, and Slip — to infer latent knowledge state.

• Multi-agent architectures are the dominant pattern. Classroom Simulacra, the AICademic pilot project (with teacher, student and instructional designer agents), Agent4Edu and SimClass all decompose the classroom into role-specific agents. SimClass includes a Teacher Agent, an Assistant Agent and Classmate Agents with defined personality traits, and evaluates agent teaching style using the Flanders Interaction Analysis System.

• Reflection and memory are treated as the core mechanisms of realistic simulation. Classroom Simulacra's Transferable Iterative Reflection (TIR) module has a reflective agent predict, compare against ground truth, write a short reflection, and hand it to a novice agent that re-predicts — looping until accuracy plateaus, with the most helpful reflections stored in a repository. Agent4Edu defines three memory operations (retrieval, writing, reflection), with summary reflection and corrective reflection occurring only in long-term memory.

• Removing contextual memory damages simulation quality. EduAgent's ablation studies showed that removing past cognitive states from memory significantly reduced performance. EduAgent triggers four parallel outputs per lecture slide — gaze, motor action, cognitive state and test response — each stored in its own memory.

• Three knowledge modelling strategies trade off differently (Table 1). Direct prompting is easy to implement and highly customisable but relies heavily on prompt quality and may produce overly idealised behaviours. Knowledge tracing offers strong predictive performance and real-time adaptation but is technically complex, data-hungry and hard to interpret. Knowledge graphs and heuristics are highly interpretable but need extensive manual curation and struggle with dynamic learning.

• Deep knowledge tracing performance varies widely across datasets (Table 2). On the seven public datasets in the PyKT benchmark (Statics2011, AS2009, AS2015, AL2005, BD2006, NIPS34, POJ), AKT achieved the strongest reported results on Statics2011 (0.8305 long interactions / 0.8466 small interactions) and AS2009 (0.7781 / 0.7878). On POJ, DKT scored 0.5979 / 0.6629, DKT+ 0.6045 / 0.6782 and AKT 0.6137 / 0.6949.

• LLM-integrated knowledge tracing outperforms prior methods (Table 3). In the reported comparison across Assist2009, Assist2015, Junyi and Nips2020, LLM-KT reached AUC 0.8870 / accuracy 0.8168 on Assist2009, 0.9356 / 0.9185 on Assist2015, 0.9018 / 0.8294 on Junyi and 0.8291 / 0.7561 on Nips2020. Relative improvements over the next best methods were +5.68 / +7.59, +2.90 / +1.02, +2.00 / +2.76 and +5.08 / +10.06 respectively. GPT-4o was reported at accuracy 0.7274 on Assist2009 and 0.6694 on Nips2020.

• LLMs can complement, not replace, human responses in item calibration. An Item Response Theory study tested six models — GPT-3.5, GPT-4, LLaMA 2, LLaMA 3, Gemini-Pro and Cohere Command R Plus — on 20 university-level algebra questions from the OpenStax textbook. Some LLMs such as GPT-3.5 performed highly but exhibited narrower zones of the Zone of Proximal Development, limiting their ability to reflect human variability. Combining human and synthetic responses at a 1:1 ratio resulted in more accurate item calibration.

• Knowledge state can be simulated at fine granularity. In TeachTune, a Personalized Reflect-Respond prompting pipeline runs three consecutive prompts — Interpret, Reflect and Respond — to maintain a learner profile across turns. Generative Students samples mastered, confused and unknown knowledge components to craft learner personas, and other work simulates varying general ability levels from low to high.

• Personality frameworks are being layered onto simulated learners. The review notes integration of the Big Five (McCrae and John) and the Myers-Briggs Type Indicator (Carlson) into generative agents to model non-cognitive dimensions of learning.

• Persistent concerns remain. The paper repeatedly flags algorithmic bias, evaluation reliability, alignment with educational objectives, limitations in open-access training datasets, and the risk of generating overly idealised or homogenised behaviours, plus the difficulty of representing genuinely diverse learners.

Methodology in Plain English

The authors conducted a thematic review rather than a quantitative meta-analysis. They ran a systematic search in Scopus, Web of Science, IEEE Xplore, ACM Digital Library and Google Scholar, using terms such as "Simulated Students", "educational simulation", "digital twin", "generative agents in classrooms" and "student Modelling with LLMs". Google Scholar was included to catch very recent work not yet indexed elsewhere.

They prioritised peer-reviewed articles published between 2021 and 2025, but also included significant preprints because the field moves quickly. Studies qualified if they proposed a method or architecture for simulating students with LLMs, applied LLMs in simulated educational settings experimentally or conceptually, or addressed cognitive, affective or personality dimensions in generative educational agents. No restriction was placed on study type.

Two independent reviewers screened the records: after removing 408 duplicates, they filtered 808 titles, resolving disagreements by discussion and consensus. Historically important articles were added manually for context even when they did not meet the inclusion criteria. Findings were synthesised qualitatively — the authors deliberately chose not to quantify results, instead identifying patterns, conceptual tensions and emerging research lines. As an accuracy check, they ran a double-verification process using the Gemini and ChatGPT LLMs. The authors also report the limitations of this approach: a possible overrepresentation of studies with positive results, restriction to English-language publications, and exclusion of grey literature such as theses, unpublished institutional documents, developer technical blogs and community-of-practice content.

Why This Matters

Impact on research. The review gives the field a shared vocabulary and structure. It connects two research traditions that usually develop separately — knowledge tracing, which originated before LLMs, and generative agent simulation — and argues the former is the natural cognitive engine for the latter. By shifting the task from predicting a correct/incorrect label to generating a response (an explanation, a solution, or a plausible error) coherent with an internal knowledge state, a knowledge tracing model becomes a student simulator. It also highlights a validation deficit inherited from the pre-LLM era, when almost half of simulated learner studies offered no formal validation.

Real-world applications:

• Teacher training. Simulated classrooms, as in SimClass, let educators rehearse instruction and receive feedback on teaching style without risking real students.

Authors’ abstract

Simulated Students offer a valuable methodological framework for evaluating pedagogical approaches and modelling diverse learner profiles, tasks which are otherwise challenging to undertake systematically in real-world settings. Recent research has increasingly focused on developing such simulated agents to capture a range of learning styles, cognitive development pathways, and social behaviours. Among contemporary simulation techniques, the integration of large language models (LLMs) into educational research has emerged as a particularly versatile and scalable paradigm. LLMs afford a high degree of linguistic realism and behavioural adaptability, enabling agents to approximate cognitive processes and engage in contextually appropriate pedagogical dialogues. This paper presents a thematic review of empirical and methodological studies utilising LLMs to simulate student behaviour across educational environments. We synthesise current evidence on the capacity of LLM-based agents to emulate learner archetypes, respond to instructional inputs, and interact within multi-agent classroom scenarios. Furthermore, we examine the implications of such systems for curriculum development, instructional evaluation, and teacher training. While LLMs surpass rule-based systems in natural language generation and situational flexibility, ongoing concerns persist regarding algorithmic bias, evaluation reliability, and alignment with educational objectives. The review identifies existing technological and methodological gaps and proposes future research directions for integrating generative AI into adaptive learning systems and instructional design.