From Content Generation to Learning Support: Pedagogy-Guided Generative Video Tutors for STEM Learning

AI summary

Overview

Research area: Natural language processing and AI for education, specifically pedagogy-guided generative video for STEM instruction.

Technical level: Intermediate. The work combines LLM-driven storyboard planning, code-centric animation generation (Manim), multimodal verification, and adaptive remediation; readers need some familiarity with LLM pipelines and video generation, but the pedagogical framing is explained from first principles.

Scope: The paper introduces PIVOT, a three-stage framework that reframes educational video generation from passive content production into a learner-centered workflow spanning instructional planning, verified video rendering, and misconception-aware assessment.

What This Paper Is About

Generative AI can already produce visually coherent educational videos at scale, but existing systems optimize for content fidelity and visual fluency rather than for learning. The authors argue that these videos typically lack explicit pedagogical structure, reliable quality control, and any mechanism for checking learner understanding or correcting misconceptions. The goal of PIVOT is to integrate pedagogy into the entire generation pipeline so that an AI-generated video becomes one link in a teaching loop that also includes assessment and targeted corrective feedback.

Key Contributions

1. A pedagogy-guided instructional video tutoring framework (PIVOT) that unifies storyboard-based planning, verified multimodal video generation, and assessment-driven remediation into a single pipeline for STEM learning.
2. A pedagogical verification harness with three components — Layout Consistency, Factual Consistency, and Cognitive Clarity — that detects and repairs failures in rendered animations so that output stays factually correct, visually clear, and pedagogically aligned.
3. An adaptive assessment and remediation stage in which quizzes are generated from the instructional storyboard without reusing the video's examples, equations, or numerical values, and incorrect learner responses trigger a targeted remediation video contrasting the wrong reasoning with the correct reasoning.
4. Empirical and expert validation across four STEM domains using automatic VLM-based scoring, ablation studies, and a questionnaire study with 32 STEM instructors, showing gains in video quality, narration quality, and perceived instructional effectiveness.

Main Findings

• Video quality (1–10 scale, Gemini-2.5-Flash at T=0): PIVOT scored 9.10 on Layout and 6.06 on Richness for an overall 7.58, compared with TheoremExplainAgent (9.00 / 4.23 / 6.62) and Code2Video (8.18 / 4.22 / 6.20). The largest margin is on Richness.
• Narration quality (0–1 scale, GPT-4o): PIVOT scored 0.87 on Accuracy and Depth and 0.91 on Logical Flow for an overall 0.89, versus TheoremExplainAgent (0.85 / 0.82 / 0.84) and Code2Video (0.78 / 0.72 / 0.75).
• Layout verification matters most for visual quality: removing Layout Consistency dropped the Layout score from 9.10 to 7.87, with overlapping elements and inconsistent organization becoming more likely.
• Factual verification affects conceptual correctness: removing Factual Consistency lowered the Accuracy and Depth score from 0.87 to 0.85; the paper gives chemical equation balancing as an example where the model may misdescribe conservation of mass as conservation of atom counts.
• Cognitive verification affects instructional coherence: removing Cognitive Clarity lowered the Logical Flow score from 0.91 to 0.89, producing abrupt topic transitions and weaker instructional progression.
• Expert evaluation favors the full workflow: with 32 STEM instructors (3 physics, 12 mathematics, 5 chemistry, 12 computer science) on a five-point Likert scale, PIVOT consistently outperformed the baseline across nearly all dimensions, with average gains of +1.67 points in Physics and +0.76 points in Computer Science. Improvements were smaller in Mathematics and Chemistry, which the authors attribute to those topics being more symbolic and rule-driven.
• Error Explanation showed the largest improvement, indicating that misconception-aware remediation contributed most to perceived conceptual understanding; Clarity scores were also higher for PIVOT.
• 96.9% of participating experts reported that the "instructional video → assessment → targeted remediation" workflow provides a more effective learning experience than instructional videos alone.
• Case studies comparing Code2Video, TheoremExplainAgent, and PIVOT on the Newton–Raphson Method, Transition Metals, and Bubble Sort show more organized layouts and richer multimodal presentation from PIVOT.

Methodology in Plain English

The framework mirrors how a human instructor prepares and delivers a lesson, split into three stages.

Stage 1 — Pedagogy-guided instructional planning. Given a learner-specified topic, an LLM produces a structured storyboard rather than code. The storyboard contains learning objectives, prerequisite activation, concept explanations with scaffolding strategies, a visual representation plan, worked examples, and diagnostic assessment probes, plus a list of shots each with a local goal, narration, visual description, visual elements, and duration. These components are grounded in established educational theory (Bloom, Sweller, Wood et al., Mayer, Renkl, Black and Wiliam).

Stage 2 — Video generation with a pedagogical verification harness. The storyboard drives executable Manim code generation, with constrained layout prompting to control spacing, font hierarchy, and animation sequencing. A multimodal verification harness then checks each shot along three axes: Layout Consistency (overlap, spacing, readability, information density), Factual Consistency (formulas, symbolic transitions, algorithmic procedures), and Cognitive Clarity (logical consistency, instructional progression). Failures trigger iterative low-temperature code repair conditioned on execution traces and rendering feedback, using GPT-5.2 as the primary generation and repair model and transferring persistent failures to Claude Sonnet 4.6. A text-to-speech module then generates narration synchronized with the rendered animations.

Stage 3 — Adaptive assessment and remediation. Quizzes are generated from the storyboard's shot-level content, deliberately avoiding reuse of the video's examples, equations, or numerical values so as to test conceptual understanding rather than memorization. When a learner answers incorrectly, the system conditions a remediation video on the quiz question, the candidate options, the learner's response, and the correct answer, producing a contrastive explanation of the specific misconception.

Evaluation setup. Two state-of-the-art baselines, TheoremExplainAgent and Code2Video, were compared on the same 40 STEM topics — 10 knowledge points each in mathematics, physics, chemistry, and computer science. Automatic evaluation used Gemini-2.5-Flash for visual dimensions and GPT-4o for narration dimensions. A separate questionnaire study with 32 instructors used randomly shuffled presentation order and five-point Likert items covering clarity, engagement, narration pace, logical flow, learning efficiency, and overall teaching effectiveness.

Why This Matters

Impact on research. The paper argues for a shift in how generative educational systems are evaluated: not only by the quality of the artifacts they produce, but by how they support teaching practice, learner understanding, and corrective feedback. It also shows that pedagogical constraints can be operationalized as verifiable checks inside a generation pipeline rather than applied only as post-hoc review.

Real-world applications:

• Scaling STEM lesson production for schools and universities where instructors lack time to prepare instructional materials and record lectures.
• Automated formative assessment with targeted remediation videos for learners who answer quiz questions incorrectly.
• Teacher-facing authoring tools that produce editable storyboards and pedagogically verified animations rather than opaque finished videos.
• Supplementary instruction in domains such as mathematics, physics, chemistry, and computer science where abstract or procedural concepts benefit from visualization.

Industry relevance. The pipeline touches several commercial areas: educational technology platforms, LLM-based content authoring tools, code-generating animation systems, and AI tutoring products. It also introduces a practical cost consideration, since the multi-stage pipeline is substantially more expensive than direct multimedia synthesis.

Future Directions

• Long-term learner modeling. The current system adapts only to immediate responses; the authors identify knowledge tracing, learner memory, curriculum progression, and individualized difficulty estimation across extended sessions as unaddressed.
• Generation efficiency. The multi-stage storyboard, rendering, verification, and remediation pipeline costs considerably more computation than direct synthesis, and reducing that cost while preserving pedagogical quality remains open.
• Human oversight in deployment. The ethics statement notes that generated educational content may still contain factual inaccuracies or pedagogical limitations, implying a need for educator review workflows.
• Generalization beyond the evaluated scope. The study covers four STEM domains and 40 knowledge points with smaller expert gains in symbolic, rule-driven subjects, raising the question of how well the approach transfers to other disciplines and learner populations.

Target Audience

This paper is most useful to researchers in AI for education and NLP who work on generative multimedia for learning, to instructional designers and educational technologists interested in embedding pedagogical principles into automated pipelines, and to developers building LLM-based tutoring or course-authoring systems. STEM instructors and curriculum specialists evaluating the reliability of AI-generated instructional video will also find the expert evaluation and rubric design informative.

Authors’ abstract

Generative AI enables scalable production of educational videos, but current systems largely focus on producing visually coherent content rather than supporting learning. As a result, generated videos often lack explicit pedagogical structure, reliable quality control, and mechanisms for assessing learner understanding or addressing misconceptions. In this work, we introduce PIVOT (Pedagogy-guided Instructional VideO Tutoring), a generative video tutoring framework for STEM learning via learning-centered instructional support.1 Inspired by conventional teaching workflows, our framework integrates pedagogy into the full generation pipeline: it first uses instructional principles to guide storyboard generation, then produces verified multimodal videos through code-centric generation and a pedagogical verification harness, and finally connects videos with assessment and misconception-aware remediation. Experiments and expert evaluations across four STEM domains show that our framework produces educational videos with pedagogically aligned content, clear and engaging presentation, coherent instructional flow, and perceived effectiveness for learning. These findings suggest a human-centered perspective on educational content generation: generative systems should be evaluated and designed not only by what they produce, but also by how they support teaching practices, learner understanding, and corrective feedback.