AI for Games in the Foundation Model Era

AI summary

Overview

Research area: Artificial intelligence for games, spanning game-playing agents, player and game modeling, game design, game development and maintenance, runtime content generation and adaptation, and automated testing.

Technical level: Intermediate. The paper is a survey rather than a new system, and it assumes familiarity with reinforcement learning, self-play, procedural content generation, and large pretrained models, while explaining each role in structured prose.

Scope (one sentence): The paper organizes AI-for-games research into six roles defined by how a system's output is immediately used, and compares what each role supplies versus what AI learns, what transfers or can be reused across settings, and what evidence actually supports the claims made.

What This Paper Is About

Foundation models, together with learned game-world models, are now used across the whole game lifecycle, not just to play games: they model players and game dynamics, help design and build games, adapt experiences at runtime, and evaluate the results. These directions have largely grown as separate research threads, which makes it hard to tell which capabilities carry over between games, engines, interfaces, and player populations, and which stay tied to one setting. The goal of the survey is to connect these threads through the outputs that pass between them, and to compare the empirical evidence each role offers at its point of use.

Key Contributions

1. A six-role taxonomy organized by the immediate use of AI output, rather than by model architecture: AI that Plays and Acts, Models Players and Games, Designs Games, Builds and Maintains Games, Generates and Adapts at Runtime, and Tests and Evaluates Games. Role assignment follows the output on which a system's principal empirical claim rests, with secondary roles cross-referenced.

2. Three recurring analytical questions applied to every role: Boundary (what structure is supplied by the game or workflow and what is assigned to AI), Transfer and Reuse (which capabilities transfer, which artifacts can be reused, and what remains setting-specific), and Evidence (what claims are supported by evaluation at the point of use).

3. A cross-role analysis of artifact exchange, tracing how trajectories can train world models, learned environments can supply experience for agents, design specifications can drive executable implementations, and feedback from play or testing can guide revision.

4. A comparative account of evaluation strength across roles, distinguishing areas where evaluation is standardized and execution-grounded (bounded game playing and selected learned environments) from areas where it is less established (persistent state in learned worlds, repeated software revision, validated player modeling, sustained runtime adaptation, representative automated testing).

The paper is accompanied by a project page and GitHub repository of the same name, and includes a system index in Appendix A recording primary and secondary roles for systems and named components.

Main Findings

• Broad pretraining expands available interfaces without removing game-specific structure. A shared visual encoder may transfer across changes in appearance more readily than a controller transfers from discrete buttons to camera-relative mouse movement, and a language goal can be portable semantically while the action grammar remains game-specific.

• Algorithm reuse, parameter sharing, and test-time adaptation are distinct claims. DreamerV3 uses one training configuration across more than 150 tasks while learning separate models and policies for those tasks. Multi-Game Decision Transformers were trained on 41 Atari games and evaluated by fine-tuning on five held-out games. Procgen varies levels procedurally across 16 game-like environments. Table 2 separates these settings from per-game training and formal-specification reuse (General Game Playing, GVGAI).

• The held-out unit matters. A new map usually changes layout while retaining controls and rules, whereas a new mode can change rewards, opponents, or transition rules within the same title; holding out an entire game tests a broader change, but success may still concern shared navigation or object-use skills rather than unfamiliar rule reasoning.

• Gameplay increasingly supplies data and feedback beyond an agent's final score. A trajectory can become simulator training data, an experiment inside a learned environment, or diagnostic evidence for software repair, and its value depends on the receiving task.

• Interface choices change what is being measured. Cradle standardizes observation and control around screenshots plus keyboard and mouse across games and applications, whereas Orak uses a structured MCP interface for plug-and-play evaluation across 12 games; these simplify comparison but support different control claims.

• Dynamics identification and demonstration retrieval address different adaptation problems. REGENT retrieves state–action examples from demonstrations of a held-out game and combines them with a pretrained policy without fine-tuning. Twin builds an executable hypothesis of unknown dynamics and searches within the resulting digital twin. Code World Models synthesize executable transition, legality, and termination functions from natural-language rules and initial offline trajectories, refining the model before competitive play and holding it fixed during it.

• Memory reliability depends on the memory unit. Episodic records and summaries help an agent act under partial observation, but stale summaries, conflicting episodes, and retrieval that misses a reset or rule change make them unreliable. EMemBench evaluates trajectory-grounded episodic memory through programmatically generated questions with verifiable answers, and GameVerse tests whether reflection on gameplay video improves subsequent play.

• Small-scale ablations limit conclusions. AgenticSTS compares memory components within Slay the Spire 2 using small, ten-run conditions, which the authors describe as supporting exploratory comparisons rather than a general advantage for bounded memory. FlashAdventure tests a different demand across 34 adventure games, where clues discovered in one scene must inform later object use and navigation.

• Coordination problems differ from partner-inference problems. Overcooked distinguishes coordinated task execution from compatibility with people; Other-Play and Fictitious Co-Play address partner variation without assuming a single shared convention; maximum-entropy population-based training trains the final agent against a diversified pool without human data; Hidden-utility self-play models human biases as hidden reward functions. ProAgent adds online interpretation of a partner's likely intentions, and systems such as MindAgent and VillagerAgent make more of the joint plan explicit through task allocation.

• Progress remains task-dependent. Stronger shared benchmarks exist for bounded play than for sustained creation, adaptation, and human experience.

• Control schemes, rules, engine interfaces, state representations, and player contexts often remain setting-specific, so downstream capability claims require validation in their target setting. Note that the provided excerpt is truncated partway through Section 3.4, so the detailed evidence for the design, build-and-maintain, runtime, and testing roles is not visible here.

Methodology in Plain English

This is a literature survey, not an experimental paper. The authors reviewed AI-for-games research and sorted it by what a system's output is used for at the moment it is produced. Actions and plans go to Play and Act; predictions about players or game dynamics go to Models Players and Games; content or rule proposals judged for design quality go to Design; executable artifacts judged for implementation quality go to Build and Maintain; session-specific outputs judged by their effect on a live, player-facing experience go to Generate and Adapt at Runtime; and test traces or judgments go to Test and Evaluate.

They then asked the same three questions of each role: what the game supplies versus what the AI must supply, what transfers or can be reused, and what evidence backs the claim. Older symbolic and learned systems are included where they clarify where a task structure or evaluation problem came from, and foundation-model systems are treated as an analytical lens rather than an inclusion criterion. The paper maps these roles and their connections in figures, situates representative systems and benchmarks by publication year and primary role, and provides per-role tables listing outputs, applications, and the principal empirical claim. A cross-setting analysis (Section 9.3.1) distinguishes evaluation settings and their supporting evidence, and Appendix A, Table 15, compares system-level interfaces, adaptation requirements, and evaluation settings.

Why This Matters

Impact on research. The survey supplies a shared vocabulary for comparing systems that are usually evaluated in different communities. It separates achievements that are often conflated — proposing an interesting mechanic, implementing it correctly, predicting a player accurately, and verifying that a prediction improved the experience — and it argues that sharing a pretrained backbone or passing an artifact between components does not by itself demonstrate transfer.

Real-world applications:

• Engine-connected development tools, such as Playco's reported use of the same pretrained model inside Playbot to create playable prototypes, and DreamGarden's development of a high-level idea into a hierarchical plan that designers can inspect and revise while specialist modules generate assets and code.
• Development and repair workflows that inspect their own output, such as Play2Code connecting a coding agent to a browser-based playtester that interacts with the running game and supplies observations for further revisions.
• Player-facing interaction systems, such as IF:CARGO, where players express rules in natural language and a language model translates them into constrained commands the engine validates and executes, and GameNGen, which learns environment responses from gameplay trajectories to produce an interactive simulator.
• Team-based game AI and autonomous testing, spanning coordination in commercial titles and automated playtesting that examines how different play styles expose different aspects of the same game.

Industry relevance. The paper's central practical point is that capability claims must be re-validated in the target setting: control schemes, rules, engine interfaces, state representations, and player contexts often stay game-specific. For studios and engine developers, this means that benchmark success in bounded play does not by itself predict success in sustained creation, runtime adaptation, or validated player modeling.

Future Directions

• Transfer across roles with re-established evidence. A central challenge the authors identify is enabling outputs and capabilities to be reused or transferred across roles while re-establishing evidence for their effectiveness in the specific structures, interfaces, and player contexts where they are ultimately used.

• Strengthening evaluation where it is weakest. Persistent state in learned worlds, repeated software revision, validated player modeling, sustained runtime adaptation, and representative automated testing remain less established than bounded game playing and selected learned environments.

• Better separating claims about generality. Comparisons would be more informative if systems stated which services they receive, since treating every successful task completion as equivalent hides the difference between acting through a semantic API and acting through native keyboard and mouse control.

• Improving memory and adaptation reliability. Stale summaries, conflicting episodes, and retrieval that misses a reset or rule change remain open problems, as does grounding adaptation in executable hypotheses or engine state when exact legality and termination matter.

Target Audience

Researchers and graduate students working on game AI, reinforcement learning, interactive agents, procedural content generation, or LLM-based agent systems who want a structured map of the field. It is also useful for game developers and technical leads evaluating whether a foundation-model technique will transfer to their engine, interface, or player population, and for benchmark designers who need to understand why evaluation strength varies so much across the game lifecycle.

Authors’ abstract

Foundation models, alongside advances in learned game-world models, are reshaping AI across the game lifecycle. Beyond playing games, recent systems model players and game dynamics, support design and development, adapt player-facing experiences at runtime, and evaluate resulting artifacts. Yet these directions have evolved largely separately, obscuring which capabilities transfer across settings and which remain tied to particular games, engines, interfaces, or player populations. We organize the literature into six roles according to the immediate use of AI output: playing and acting; modeling players and games; designing games; building and maintaining games; generating and adapting at runtime; and testing and evaluating games. For each role, we examine what structure is supplied by the game or workflow, what AI learns or produces, which capabilities and artifacts transfer across settings and roles, and what evidence supports the claims. We identify cross-role connections: trajectories train world models, learned environments provide experience for agents, design specifications drive executable implementations, and play or testing feedback guides revision. However, control schemes, rules, engine interfaces, state representations, and player contexts often remain setting-specific, so downstream claims require validation in the target setting. Evaluation is most standardized for bounded game playing and selected learned environments, while persistent state in learned worlds, repeated software revision, validated player modeling, sustained runtime adaptation, and representative automated testing remain less established. The central challenge is to reuse or transfer outputs and capabilities across roles while re-establishing evidence for effectiveness in the game-specific contexts where they are used.