TEAS: Trusted Educational AI Standard: A Framework for Verifiable, Stable, Auditable, and Pedagogically Sound Learning Systems

AI summary

Overview

Research area: AI safety and ethics applied to education — specifically evaluation standards and deployment-readiness criteria for AI tutoring and learning systems.

Technical level: Intermediate. The framework itself is described conceptually and is readable by non-engineers, but the paper assumes familiarity with concepts such as retrieval grounding, hallucination, non-determinism, and model auditing, and it includes a small empirical comparison in an appendix.

Scope: The paper proposes TEAS (Trusted Educational AI Standard), a four-pillar deployment-readiness standard — Verifiability, Stability, Auditability, and Pedagogical Soundness — arguing that trustworthiness comes from system architecture rather than raw model capability, and supports that claim with a controlled appendix comparison of a grounded 8B model against larger baselines.

What This Paper Is About

AI adoption in education has raced ahead on capability while lagging on trustworthiness, and existing evaluation approaches are split into silos: institutional policy checklists with no technical verification, pedagogical frameworks that simply assume the AI works reliably, technical metrics that ignore educational context, and vendor disclosures based on self-attestation. The goal of this paper is to define what it actually means for an educational AI system to be ready for deployment, by unifying verifiability, stability, auditability, and pedagogical soundness into one integrated standard. The core argument is that safety comes from deliberate architectural scaffolding around a model — as in Khan Academy's Khanmigo build-out around GPT-4 — not from the base model itself.

Key Contributions

1. The TEAS framework — an integrated deployment-readiness standard built on four interdependent pillars: Verifiability (grounding content in authoritative, citable sources), Stability (deterministic core knowledge across phrasings, sessions, and versions), Auditability (independent institutional inspection of knowledge, reasoning, and bias), and Pedagogical Soundness (Socratic questioning, scaffolding, spoiler prevention, and strategies such as spaced repetition, retrieval practice, and metacognitive prompting).

2. A gap analysis of existing frameworks — Table 1 compares policy frameworks (1EdTech), pedagogical frameworks (UNESCO), technical metrics (hallucination indices), and vendor disclosures (LearnLM), showing that each covers pillars in isolation and none covers all four. TEAS is presented as the only entry marked Yes across Verifiability, Stability, Auditability, and Pedagogy.

3. A taxonomy of documented failures in educational AI — factual and reasoning failures (hallucination, "sneaky errors"), pedagogical misalignment ("cognitive offloading," "vaporized learning"), invisible errors and trust miscalibration, instability and non-determinism, black-box opacity, and algorithmic bias and equity concerns.

4. An empirical case study (Appendix A) — a controlled comparison claiming that an 8-billion parameter open-source model (Qwen3-8B) with structured knowledge graph grounding outperformed models up to 15× larger on criteria derived from the TEAS pillars.

Main Findings

• Capability is not the same as deployability: The Khanmigo case is presented as the central evidence — using GPT-4 as a tutor required a separate calculator tool for numerical operations, visual preprocessors to convert mathematical graphics into text, and re-engineered multi-path reasoning to follow unconventional student solutions. Reliability resided in the scaffolding, not in GPT-4.

• Failures are interconnected, not isolated: The paper argues a factual hallucination becomes a pedagogical crisis when students cannot detect the error; a pedagogically sound system becomes unreliable when its knowledge base is unstable; an accurate system becomes ungovernable when institutions cannot audit it. This is why partial compliance is treated as insufficient.

• Stakeholder frameworks do not connect: Policy checklists (1EdTech AI Preparedness Checklist, Athena Infonomics checklist) ask vendors the right questions but provide no standardized method to verify answers. Pedagogical frameworks (UNESCO AI Competency Frameworks, CAIAF, HCAIF) assume reliability. Technical metrics (relevance, hallucination, toxicity, the Unified Explainability Score) are domain-agnostic. Vendor efforts (OpenAI System Cards, Google's LearnLM in Gemini 2.5 Pro, 1EdTech's TrustEd Apps rubric) are self-attestation rather than independent audit.

• Pedagogical soundness is a deployment blocker, not a nicety: During the peak of generative AI adoption in 2023–2024, institutions could not confidently assign homework or take-home essays, and multiple school districts and universities considered or implemented outright bans on tools like ChatGPT because of pedagogical trust, not technical capability.

• Architecture changes the economics: Because trustworthiness is argued to come from design, affordable open-source and sovereign models (for example from Sarvam AI and AI4Bharat) could meet deployment standards. The paper cites a target of roughly Rs. 30 (~$0.34) per student per year for deployment at scale in India (~150M students), economics described as impossible with frontier models.

• Empirical result in Appendix A: Across five GATE-style mathematics questions, judged blindly by an independent LLM judge (Claude Opus 4.5), the grounded 8B model averaged 13.8/15, versus 12.4/15 for an 80B model (an 11% gap) and 11.6/15 for a 120B baseline (a 19% gap). The paper states this gap was driven by superior citation quality (the appendix text is truncated at this point).

• Pillar failure modes are enumerated: Verifiable but pedagogically unsound systems cite sources while handing over answers; pedagogically sound but unstable systems confuse students with contradictory explanations; verifiable and stable but unauditable systems can embed undetectable bias; sound, stable, and auditable but unverifiable systems can teach consistent misinformation from hallucinated sources.

Methodology in Plain English

The paper is primarily conceptual and synthetic rather than experimental. The author first assembles a taxonomy of documented failure modes in educational AI from prior literature and real deployments, using the Khanmigo build-out as the flagship example of the gap between model capability and deployment readiness. Next, the author surveys four categories of existing frameworks — institutional/policy, pedagogical/ethical, technical/model-level, and vendor disclosures — and tabulates which TEAS pillars each category covers, producing the integration-gap argument. From that gap, the author defines the four pillars and, for each, specifies a definition, requirements, why it matters, and what current practice fails to provide. Finally, an appendix case study applies the framework: an 8B open-source model augmented with a structured knowledge graph (Metacog's Equation Grounder) is compared against three larger baselines on five GATE-style mathematics questions, with an independent LLM judge scoring blind on criteria mapped to the pillars.

Why This Matters

Impact on research: The paper argues for reallocating attention from capability maximization toward "trustworthiness systematization," suggesting that trustworthiness gains in smaller models may have more deployment impact than capability gains in frontier models, and that benchmarks like MMLU do not capture what deployment actually requires.

Real-world applications:

• Institutional procurement: giving review boards and procurement officers a structured rubric for evaluating systems against four pillars rather than relying on vendor claims.
• Vendor design: providing requirements to build in from the outset (hybrid systems, audit features, architecturally enforced pedagogy) instead of retrofitting trust later.
• Policy and certification: offering a foundation for mandatory compliance regimes with independent auditors, analogous to textbook review committees, medical device approval, aviation certification, and financial auditing.
• Global equity: enabling reliable AI tutoring at billion-student scale in the Global South using affordable or sovereign models where commercial pricing is prohibitive.

Industry relevance: The framework reframes TEAS compliance as a competitive advantage in a market hesitant over trust, and it argues that domain-specific educational AI standards are needed because generic AI safety rules do not capture pedagogical harm or curriculum stability.

Future Directions

• Developing the metrics and audit tools that operationalize compliance, since TEAS currently defines what trustworthiness requires but not how to measure it.
• Creating a companion standard for agentic AI security risks, which the paper explicitly excludes and tentatively names "TEAS-Security."
• Cross-disciplinary collaboration among educators, AI researchers, and policymakers, plus empirical validation of the framework across diverse contexts.
• Mechanisms for continuous monitoring and re-certification, given how quickly AI systems change.

Target Audience

Educators and instructional designers assessing whether an AI tool supports learning; school, district, and university administrators and procurement officers making deployment decisions; policymakers and regulators drafting education-specific AI rules; EdTech developers and AI researchers designing tutor systems; and AI safety and ethics researchers interested in domain-specific trustworthiness standards rather than general-purpose benchmarks.

Disclosure noted in the paper: The author is Founder and CEO of Metacog, which developed the Equation Grounder module evaluated in Appendix A. Metacog is releasing that module as open-source concurrent with publication, and the Appendix A protocol used blind evaluation with an independent third-party judge (Claude Opus 4.5). The author states the primary contribution is the vendor-agnostic TEAS framework itself.

Authors’ abstract

The rapid integration of AI into education has prioritized capability over trustworthiness, creating significant risks. Real-world deployments reveal that even advanced models are insufficient without extensive architectural scaffolding to ensure reliability. Current evaluation frameworks are fragmented: institutional policies lack technical verification, pedagogical guidelines assume AI reliability, and technical metrics are context-agnostic. This leaves institutions without a unified standard for deployment readiness. This paper introduces TEAS (Trusted Educational AI Standard), an integrated framework built on four interdependent pillars: (1) Verifiability, grounding content in authoritative sources; (2) Stability, ensuring deterministic core knowledge; (3) Auditability, enabling independent institutional validation; and (4) Pedagogical Soundness, enforcing principles of active learning. We argue that trustworthiness stems primarily from systematic architecture, not raw model capability. This insight implies that affordable, open-source models can achieve deployment-grade trust, offering a scalable and equitable path to integrating AI safely into learning environments globally.