Towards Self-Evolving Narrative Systems: Real-Time Adaptation and Long-Term Optimization with LLMs

AI summary

Overview

Research area: LLM-driven interactive narrative systems / AI storytelling and game design (cs.CL).

Technical level: Intermediate. Readers should be comfortable with prompt engineering, sentence embeddings and cosine similarity, and basic user-study statistics; no model training is involved.

Scope: A hybrid interactive-storytelling framework that combines author-written branching plots with real-time LLM generation, plus a feedback-driven loop in which a second LLM rewrites the prompts between play sessions, evaluated with a 12-participant user study on a Divinity: Original Sin 2 “Escape from Fort Joy” scenario.

What This Paper Is About

Interactive stories face a trade-off: hand-authored branching plots keep a story coherent but give players little real freedom, while fully LLM-generated stories give freedom but drift into contradictions, inconsistent characters, and lost authorial control. The paper proposes a system that keeps a pre-authored narrative backbone, switches to real-time generation only when a player's action diverges from it (measured by semantic similarity), and then automatically rewrites its own prompts based on player ratings and comments collected after each playthrough. The goal is a narrative system that stays coherent within a session and measurably improves across sessions without manual prompt engineering.

Key Contributions

1. Modular narrative structure with a semantic alignment router. Human-written Game Scripts are expanded offline into a hierarchical tree of reusable nodes containing Scripted Branches (canonical storyline) and Pre-generated Branches (LLM alternatives). At runtime, a cosine-similarity score between the player's input embedding and node embeddings decides between freeform exploration, following an authored branch, or generating a new one.

2. A self-evolving, feedback-driven optimization loop. An LLM Optimizer ingests per-playthrough ratings on narrative coherence, NPC consistency, and player agency plus interaction logs, diagnoses recurring problems, and writes concrete textual patches to the three prompts (System, Scripted, Generated), which are then redeployed — a closed loop using in-context learning rather than fine-tuning.

3. A human-centered evaluation isolating each system. Two sequential within-subject tracks with the same 12 participants: Track A compares the structured Game Plot system against a flat single-LLM-call baseline; Track B compares the optimized system against its static counterpart (Track A's structured system).

4. Released artifacts. The authors state they will release code and the Fort Joy scenario specification for reproducibility.

Main Findings

• Track A — structure improves coherence at little agency cost. Compared with the flat baseline, the structured system raised narrative coherence by +0.5 and NPC consistency by +0.5, with a modest decrease in player agency of –0.2.
• Track A — secondary measures all rose. Immersion 4.0 → 4.3 (+0.3), handling of unexpected inputs 3.6 → 4.1 (+0.5), willingness to replay 3.4 → 3.8 (+0.4), all on 5-point Likert scales.
• Track A — qualitative theme shifts. Memory failures fell from 9/12 to 2/12, Consistent Persona rose from 2/12 to 11/12, Opaque Endings fell from 7/12 to 3/12, Earned Closure rose from 1/12 to 9/12, and Semantic Adaptation rose from 1/12 to 9/12. Choice Stagnation increased from 1/12 to 4/12.
• Track B — optimization yields smaller but statistically significant gains. Coherence improved +0.3 (3.9 → 4.2, p < 0.05), NPC consistency +0.3 (4.3 → 4.6, p < 0.05), and player agency +0.2 (3.3 → 3.5, p < 0.1), with n = 12 × 3 within-subjects.
• Track B — the loop reduces opaque endings. Opaque Endings dropped 7/12 → 3/12 (a 57.1% reduction) and Earned Closure rose 1/12 → 9/12, because refined prompts began recognizing semantic intent (e.g., “I'm in, let's escape together”) rather than exact phrases.
• Track B — stagnation and adaptation. Choice Stagnation dropped from 8/12 to 2/12 while Semantic Adaptation rose from 1 to 9, supporting the claim that the coherence–agency trade-off is dynamic rather than zero-sum.
• A residual structural failure. Scene 7 still looped for non-canonical companion choices (participants P7 and P11), which the authors attribute to end-condition coverage that prompt refinement cannot fix.

Methodology in Plain English

The system runs in three phases.

Phase I, offline pre-generation. A human author writes a Game Script defining environments, NPCs, end conditions, and “key questions” marking pivotal decisions. An LLM Generator (gpt-4o-mini) turns each key question into multiple narrative paths made of modular nodes. Three prompt types do the work: a System Prompt holding global world rules, a Scripted Prompt producing the canonical storyline, and a Generated Prompt producing alternative branches.

Phase II, gameplay. Each player input is embedded with MiniLM-L6-v2 and compared by cosine similarity against the embeddings of available pre-generated nodes, giving a score in [−1, 1]. If the score is below θ_low = 0.40, the player stays in the current node and keeps exploring. If it is at or above θ_high = 0.70, the system jumps to the best-matching authored node. In between, it generates a new node on the fly, conditioned on scene state, NPC traits, and unresolved story elements.

Phase III, self-evolution. After a playthrough, players rate coherence, NPC consistency, and agency on 5-point Likert scales and leave free-text comments. An LLM Optimizer (gemini-2.5-flash) aggregates this with interaction logs, forms a diagnostic hypothesis (the paper's example: Fane's core motivation is underspecified in his Generated Prompt), proposes a specific textual patch, and redeploys the revised prompts for the next round.

Evaluation. Twelve participants (aged 18–36; 10 experienced RPG players, 2 professional game designers) each completed both tracks in a session of roughly 240 minutes, with a one-day break between tracks and counterbalanced condition order within each track. The stimulus was a nine-scene “Escape from Fort Joy” arc with key questions k1–k3 per scene. Surveys combined Likert ratings with open-ended questions, and two researchers performed inductive thematic analysis, resolving coding disagreements by discussion.

Why This Matters

Impact on research. The paper reframes LLM storytelling from a one-shot generation problem into a continual, player-informed optimization problem, and supplies a concrete mechanism (feedback → diagnosis → prompt patch → redeployment) that sidesteps retraining. It also gives a quantitative account of where static prompts fail: prior work such as the SCORE framework showed fixed prompts break down over long interactions, and this paper shows a lightweight loop recovers part of that loss.

Real-world applications.

• Game studios building narrative-heavy RPGs that want authored main arcs with LLM improvisation for off-script player behavior.
• AI dungeon-master or tabletop-assistant tools that need consistent NPC personas across long campaigns.
• Interactive training simulations and educational role-play, where character consistency and reliable scene closure matter more than open-ended creativity.
• Live-service or episodic games that could ship prompt updates between content releases instead of patching game logic.

Industry relevance. The approach targets a known production pain point: manual prompt engineering and manual playtesting iteration do not scale to every failure mode players discover. Automating the diagnose-and-patch step offers a cheaper path to post-launch narrative quality improvement, though the authors flag API dependence, latency (≈1.4s for GPT-4o Mini), and reproducibility risk from model updates as practical constraints.

Future Directions

• Replace lexical similarity with intent classification. The authors note that cosine similarity cannot distinguish refusal from postponement or ironic agreement, contributing to the 3/12 residual opaque endings in the final optimization round. Dedicated intent classifiers, possibly with structured knowledge representations, are proposed to decouple narrative closure from surface wording.
• Principled threshold selection. θ_high = 0.70 and θ_low = 0.40 were set heuristically from pilot observations; a principled optimization procedure would clarify how thresholds shift the authored-versus-emergent balance.
• Architectural fixes beyond prompt refinement. Scene 7 looping on non-canonical companion choices suggests end-condition coverage needs structural change, not just better prompts.
• Larger, broader, and more continuous evaluation. The 12-participant single-genre pilot limits generalizability; the authors call for expert-based assessment, real-time (rather than round-based) deployment to capture intra-session adaptation, formal inter-rater reliability, and locally fine-tuned models with hierarchical context management and persistent long-term memory.

Target Audience

Most useful to researchers and practitioners working on LLM-based interactive narrative, procedural story generation, and AI game design — particularly those building systems that must balance authored structure against player freedom. It is also relevant to HCI and games-user-research audiences interested in mixed-methods evaluation of generative systems, and to narrative designers curious about how automated feedback loops compare with manual playtesting iteration. Readers wanting a full training-level treatment of LLMs will not find it here; the contribution is architectural and empirical.

Authors’ abstract

Traditional interactive narrative systems offer strong authorial control and structural coherence but often restrict player agency. In contrast, LLM-driven narratives enable open-ended generation and greater flexibility, yet frequently suffer from inconsistency and diminished narrative control. To navigate this trade-off, we propose a hybrid narrative framework that dynamically balances pre-authored content with real-time LLM generation via semantic similarity alignment. Additionally, sustaining narrative quality over extended gameplay remains a core challenge in LLM-based storytelling. To address this, we introduce a self-evolving, feedback-driven optimization framework that leverages in-context learning to iteratively refine prompts, eliminating the need for manual prompt engineering. Through a case study and participant-based evaluation, we show that our framework enhances narrative coherence while preserving player agency, effectively aligning generated content with both authorial intent and evolving player behavior.