VibeGame: Prompt-to-Game Development with AI-Native Engine and Self-Evolving Adversarial Agent Team

AI summary

Overview

Research area: Multi-Agent Systems, applied to automated software and game development.

Technical level: Advanced. The paper assumes familiarity with LLM coding agents, agent harnesses, game engines, and benchmark-based evaluation of generated software.

Scope: VibeGame is a framework that turns a natural-language prompt (plus optional reference images) into a complete, playable 2D game project, using a text-first game engine, an eight-role adversarial agent team, and a training-free mechanism that recycles experience from accepted projects.

What This Paper Is About

Building a playable game means coordinating code, scenes, art assets, animations, physics, and configuration files, then checking that all of it works when someone actually plays. Existing LLM coding agents can write software, and world models can generate interactive video, but neither produces a game project that can be built, inspected, versioned, played, and revised. The paper's goal is to make prompt-to-game (P2G) development work end to end, where agents can freely inspect and control the running game during development rather than only being graded at the end.

Key Contributions

1. VibeGame framework. A system that transforms natural-language prompts into complete playable game projects and supports later iterative editing of those projects.

2. AI-native game engine. An engine built on Phaser that satisfies three stated properties: GUI-Independent (the whole project is plain-text JSON scenes and nodes that agents edit directly), Runtime-Accessible (frame-synchronous control lets agents pause, step forward frames, inject actions like "press jump", inspect node positions and velocities, and capture screenshots), and Source-Available (the engine's uncompiled source is copied into the project so agents can read exact APIs).

3. Adversarial Agent Team (AAT). Eight specialized roles — orchestrator, designer, artist, architect, programmer, auditor, player, and reviewer — that separate implementation from review across three stages: Intention Alignment, Parallel Development, and Adversarial Correction.

4. Training-free self-evolution. After a project is accepted, the system distills it into reusable artifacts of three kinds: skeletons (runnable project templates with error notes and art packs), modules (self-contained behavior scripts, sometimes with self-checking rules), and contracts (documents defining role-specific inputs, outputs, dependencies, and verification criteria), which seed future projects.

Main Findings

• VibeGame beats all baselines under the same model. On a 30-task subset of GameCraft-Bench, VibeGame in P2G outperforms every evaluated harness in either P2G or PA2G with the same underlying model, including GPT-6.1-Sol-based runs.

• Large gain with the weaker model. With GPT-5.6-Luna, VibeGame scores 49.99 overall versus 38.29 for the strongest baseline (Qwen Code, P2G), an improvement of 11.70 points, and leads in all four rubric categories.

• Smaller but real gain with the frontier model. With GPT-6.1-Sol, VibeGame scores 72.99 overall versus 68.80 for Codex (P2G), a 4.19-point gain. It trails by less than one point in Core Mechanics (86.26 vs 87.18) and Functional Visuals (62.29 vs 62.48), but leads by 4.76 in Content Depth (67.81 vs 63.05) and 7.72 in Art & Presentation (76.97 vs 69.25).

• General coding agents cluster tightly. In the PA2G setting, general coding agents score within 2.50 points of each other, and GPT-5.6-Luna does not convert image generation into a visual advantage in P2G.

• OpenGame underperforms general coding agents. Despite being purpose-built for game development, OpenGame does not beat general coding agents, which the authors attribute to limited generalization beyond its templates or poor adaptation to GameCraft-Bench's demo-recording requirements.

• Both core components matter. Ablation on a 10-task subset with GPT-5.6-Luna: removing the AI-native engine drops the overall score by 10.33 points (53.45 to 43.12) and removing the adversarial review roles of AAT drops it by 4.63 points (to 48.82). Adding the engine alone to Codex with GPT-6.1-Sol raises its overall score by 2.69 points.

• Evolved experience transfers across models. Adding experience evolved by GPT-6.1-Sol on 5 tasks outside the ablation set to VibeGame with GPT-5.6-Luna raises the overall score by 2.92 points (to 56.37), suggesting experience transfers from a stronger model to a weaker one.

• Broad genre coverage. VibeGame outperforms the strongest baseline of each model on 13 of the 15 game types.

• Human raters agree with the judge. A blind user study with 30 participants, 150 game pairs, and 300 judgments gives VibeGame an average overall human win rate of 72.7% against all five GPT-6.1-Sol baselines, with wins in every rubric category.

• The judge choice matters. Switching from GPT-6.1-Sol to GPT-5.5 as judge changes both absolute scores and ranking; the paper documents three concrete GPT-5.5 judging errors, while VibeGame still achieves the best overall score under both judges.

Methodology in Plain English

The team first rebuilt the game engine itself so that an AI agent, not a person clicking menus, is the primary user. Every game is stored as readable text files describing scenes, nodes, assets, scripts, and settings, with schema validation that catches broken references before the game runs. Because everything is text, an agent can read and edit any part of the project directly.

The second piece is control over the running game. Instead of a human watching a screen and judging by eye, the agent can pause the game, advance it a fixed number of frames, change properties like a node's velocity, simulate inputs such as pressing jump, and take screenshots. This turns real-time gameplay into something an agent can inspect step by step.

On top of this engine, eight agents divide the work. A designer expands a short request into a full game design document, and an artist produces concept art and then engine-ready assets using image-generation models (GPT-Image-2 and Nano Banana Pro). The orchestrator splits the game into parallel components with a product requirements document; an architect plans each component and its verification; a programmer implements it; an auditor checks it statically for consistency with the engine and the spec; and a player actually runs it and uses a vision model to judge whether it looks right. A component counts as done only after passing both static audit and runtime play-testing. Finally an independent reviewer gates the finished project on functionality, visual quality, and playability, rejecting with specific reasons that the orchestrator turns into repair tasks, looping until acceptance.

After a project is accepted, the system harvests reusable knowledge from it without any model training: templates for that genre, standalone behavior scripts, and written protocols for how the roles should collaborate on recurring features. Future projects start from these artifacts and must revalidate them rather than trust them blindly.

Evaluation ports GameCraft-Bench from Godot to Phaser, keeping the original game descriptions and rubrics intact, sampling 2 tasks from each of 15 game types for 30 tasks total. Every harness gets 24 hours per task at xhigh reasoning effort, and GPT-6.1-Sol serves as judge at medium reasoning effort, with scores averaged over three judge passes.

Why This Matters

Impact on research. The paper argues that the bottleneck for agentic game development is not model capability but interface design: agents need a project representation they can fully inspect and a runtime they can control. It also shows a concrete path for carrying validated experience forward across independent runs without fine-tuning, which is a general problem for agents that otherwise relearn everything from scratch each session.

Real-world applications:

• Rapid prototyping of playable game concepts by small studios or solo creators without programming or art teams.
• "IP transfer" edits, where an existing game's mechanics are kept while all visuals are regenerated for a different theme, as demonstrated with a Kung Fu Panda reskin of a boss-fight project.
• "Genre transfer" edits, converting a turn-based card game into a real-time action boss fight while preserving the original cyberpunk visual identity.
• Automated regression testing of games, since the frame-synchronous interface lets agents replay scripted input traces and compare rendered output against intent.

Industry relevance. The results suggest frontier models already handle core mechanics and reasonable visuals on their own, with the remaining gap concentrated in content depth and presentation polish. That is precisely where a structured harness plus an adversarial review team adds value, which matters for anyone building agent tooling on top of commercial or open engines.

Future Directions

• 3D and long-tail games. The authors state they focus on 2D games and have not validated the framework in more complex 3D scenes or used it to develop complete commercial games that sustain long-term play.

• Porting the design principles to established engines. Bringing the AI-native properties to engines such as Godot, so agents can also operate existing game projects rather than only new ones, is named as a promising extension.

• Better judges. The documented GPT-5.5 judging errors — miscounting distinct track layouts, missing a bench in a shop UI, and failing to see moving resource particles on conveyors — raise the question of how reliably automated game evaluation can be trusted, especially with downscaled recordings.

• Richer experience reuse. The self-evolution artifacts currently come in three forms (skeletons, modules, contracts) distilled from single accepted projects; how they compound over many projects, and how they should be curated or retired, is not reported.

Target Audience

Researchers and engineers working on multi-agent LLM systems, agentic software engineering, and tool/harness design for coding agents will get the most from this paper. It is also relevant to game developers and technical artists interested in AI-assisted prototyping, and to benchmark designers who need to know how sensitive automated game evaluation is to the choice of judge model. Readers need comfort with agent architectures and game engine concepts; the paper is not an introductory treatment.

Authors’ abstract

Large language model agents have enabled software development through natural-language interaction, but extending this paradigm to end-to-end game development remains challenging. Building a playable game requires coordinating code, scenes, assets, animations, physics, and configurations, while allowing developers to validate their integrated behavior through interactive testing. We introduce VibeGame, a framework for creating complete 2D game projects from natural-language prompts and optional reference images. Its AI-native game engine represents project state as schema-validated structured text and provides frame-synchronous runtime control, enabling agents to inspect game states, execute actions, and verify behavior at their own pace. Built upon this interface, we design an Adversarial Agent Team of eight specialized agents to separate implementation from review, so that problems are found and fixed through multi-round loops. For each conducted and accepted project, we further introduce a training-free self-evolution mechanism to distill it into reusable skeletons, modules, and collaboration contracts for future development. We evaluate prompt-assets-to-game (PA2G), where agents are given an asset library, and prompt-to-game (P2G), where agents generate their own art. On a GameCraft-Bench subset, VibeGame in P2G consistently outperforms a range of agent harnesses in either P2G or PA2G with the same underlying model, including GPT-6.1-Sol.