Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness

AI summary

Overview

Research area: Agentic AI systems for game development — specifically, closed-loop game generation harnesses that combine coding agents with automated playtesting and experience-oriented revision.

Technical level: Intermediate. The paper is conceptual and readable without deep technical background, but it assumes familiarity with LLM coding agents, agentic workflows, and benchmark evaluation (task success rates, rubric scores, rollout-based testing).

Scope: A four-role recursive harness (Designer, Builder, Player, Reviewer) that iteratively refines generated games toward better player experience, evaluated on GameCraft-Bench and GameASG-Bench plus a user study.

What This Paper Is About

Coding agents can now produce games that run correctly, but the paper argues that program correctness does not ensure an enjoyable player experience — engaging mechanics, appropriate difficulty, coherent presentation, and responsive feedback are not captured by "the code executes." The authors build Recursive Game Creator, a harness that closes the loop between game construction, automated playtesting, and preference-informed revision, so a game is refined across multiple versions rather than simply made to work. The goal is to move agentic game development from rough prototypes toward product-level, entertaining games.

Key Contributions

1. Product-level recursive harness. A multi-round closed-loop game design harness connecting four roles — Designer, Builder, Player, Reviewer — that supports continued game evolution with fixed model parameters and user-directed customization.

2. Coding-native and experience-oriented game evolution. A Player that writes and executes reusable code policies through programmatic game interfaces (rather than driving the game frame-by-frame through a GUI) to collect diverse gameplay trajectories, paired with an experience-oriented Reviewer that combines shared signals and game-specific rubrics into preference-informed feedback.

3. State-of-the-art game design capability. The authors report the highest overall score on GameCraft-Bench and the highest mean runtime-check pass rate among compared methods on GameASG-Bench, with a user study supporting preference-guided game evolution.

Main Findings

• GameCraft-Bench overall score of 77.89 after three refinement rounds. On 45 selected GameCraft-Bench tasks, three rounds raise the overall score from 72.70 (round 1) to 75.35 (round 2) to 77.89 (round 3) — a gain of 5.19 points across rounds. The paper also reports this as 6.63 points over the same-model baseline (GPT-6 Astra high, baseline 71.26) and 6.37 points above the strongest reported baseline in the table.

• All five GameCraft-Bench categories improve across the authors' three rounds. Action 67.96 → 70.33 → 77.62; Timing 68.76 → 73.19 → 74.41; Strategy 72.10 → 72.96 → 74.46; Simulation 77.72 → 80.57 → 82.94; Adventure 76.95 → 79.69 → 80.00.

• Action shows the largest single-category increase (+9.66, from 67.96 to 77.62). The paper notes the first two rounds remain below the same-model baseline Action score of 73.33, and suggests action-game quality may depend on coordinated changes to controls, collision handling, combat feedback, and progression — while stating that category scores alone do not identify the cause.

• GameASG-Bench strict task success of 25/47 (53.2%), reported as a 34.1 improvement over the same-model baseline. The same-model comparison is GPT-6-Astra high with Codex CLI, which passes 9/47 tasks (19.1%). The paper describes this as a strict task success rate requiring valid delivery, completed evaluation, and passing every L1 check and every applicable L2 P0/P1 check.

• Highest mean runtime-check (L2) pass rate of 93.4% among compared methods. Alongside it, the harness reports L1 of 98.3%, L2 P0 of 99.0%, L2 P1 of 92.2%, and L2 P2 of 93.9%. The paper states the mean L2 and extended-feature (P2) scores are the highest among the reported configurations.

• Coding-native Player exploration coverage. In a first-person-shooter test scene, 48 recorded coding-native policy rollouts reach areas across the map including building interiors, with the source reported at 97.1% grid coverage including 11 interiors. The paper explicitly notes the GUI routes in the figure are illustrative, so the figure shows coverage rather than a measured speedup.

• User study shows longer playtime and higher ratings across rounds. On a 1–10 scale, controls go 2 → 5 → 8, playability 1 → 4 → 7, depth 3 → 4 → 6, and art 2 → 6 → 8 across rounds 1–3. Average playtime rises from 1.50 minutes to 4.87 minutes to 12.41 minutes.

• Qualitative evidence of multi-round refinement. Example games include Twilight Front (adds a fog-of-war display and expands its hero roster), KAZE (changing race conditions, a career garage, car liveries), See You Tomorrow (character dialogue, player choices, event art, classroom menu), and Meme Arena (three-character roster evolving in arena detail, skill effects, summon attacks, and movement guidance).

Methodology in Plain English

The harness imitates a small game studio with four roles that operate in a loop:

• Designer. Takes the user's brief, the current game, and recent feedback, and produces a structured plan: a hypothesis about the intended experiential improvement, the concrete changes to make, acceptance goals (scenes, inputs, observable outcomes), and a list of asset requests.
• Builder. Implements the plan in two stages — a playable draft, then generation of the requested assets, then integration into an integrated candidate — followed by targeted self-checks and project checks.
• Player. Instead of having a language model look at a screenshot and choose each action, the Player writes executable code policies that read the game's exposed state and choose valid actions, then runs those policies to produce gameplay trajectories. Each trajectory records states, actions, events, progress, elapsed time, and termination outcomes. The paper's argument is that this is faster, repeatable, and preserves structured state that GUI-based testing would discard. Policies are also varied in strategy, skill, exploration, and risk tolerance, so different failure modes surface.
• Reviewer. Turns trajectories plus sampled screenshots into an experience diagnosis: success rate and playtime (framed as difficulty calibration, not maximizing success), a game-specific rubric instantiated per genre, and visual checks for readability, style, and feedback. It also compares a candidate against the retained version using consistent policy settings and observation access, and outputs an A/B preference, a tie, or an "unavailable" judgment. The judgment determines which version is retained, and the review feeds the Designer's next plan.

The authors evaluate on GameCraft-Bench (Godot games, replayed gameplay plus game-specific rubrics) and GameASG-Bench (browser games graded on a source-code level L1 and a runtime level L2 with P0/P1/P2 check tiers), then add qualitative version-by-version comparisons and a user study with expert scores and playtime.

Why This Matters

Impact on research. The paper shifts the optimization target for agentic game generation from "does the artifact run and satisfy requirements" to "does the artifact produce a good experience," and it does so without updating model parameters — improvement comes from harness structure and structured preference feedback. It also proposes an evaluation design where fast automated play (code policies) is deliberately decoupled from slower visual assessment.

Real-world applications (as suggested by the paper's framing and examples):

• Automated iteration on game prototypes, refining generated games across versions rather than shipping a first-pass build.
• Genre-specific quality assurance — the paper's examples span racing, visual novels, MOBA-style games, and an FPS test scene, each with different criteria such as responsive steering, coherent dialogue, or clear track guidance.
• Personalized game tuning: users can supply explicit textual preferences during play, and preferences inferred from play inform the next design round.
• Coverage-style playtesting that reaches content such as building interiors in an FPS map that a small number of scripted routes might miss.

Industry relevance. The four-role decomposition mirrors real studio roles (design, implementation, QA playtesting, review), which makes the harness structure legible to production pipelines. Automated, repeatable playtesting that reports success rate, playtime, and failure points has direct value for QA and live-service iteration. Cheaper iteration also matters for smaller teams that cannot staff round-the-clock playtesting.

Caveats the paper itself flags: proxy policy playtime does not establish human interest or enjoyment; proxy strategies do not represent the full human player population; and the authors state that a controlled speed comparison against GUI-based collection would require matching budgets and observation access, including policy construction, repair, execution, and review costs. The paper does not report participant counts, statistical significance testing, or a controlled latency benchmark.

Future Directions

• Controlled comparison against GUI-driven playtesting. The paper states a controlled comparison must match budgets and observation access — including policy construction, repair, execution, and review — and leaves that measurement open; the exploration figure is explicitly illustrative rather than a measured speedup.
• Validating proxy policies against real players. The authors note proxy strategies provide varied test behavior but do not establish representation of the full human player population; more human trajectories and explicit feedback are needed.
• Diagnosing causes of late-stage gains. The paper observes that Action quality improved most in the third round and is consistent with multi-cycle coordination of controls, collision, combat feedback, and progression, but states that category scores alone do not identify the cause.
• Extending toward product-level games and individual adaptation. The conclusion frames the harness as a foundation for advancing functional prototypes toward product-level games and for evaluating continued adaptation to individual players, and notes code is coming soon.

Target Audience

Readers who will get the most from this paper: researchers working on agentic code generation, LLM-based game generation, and automated software evaluation; game developers and technical directors interested in automated playtesting and iterative content refinement; and benchmark designers who need to measure experiential quality rather than functional correctness. The paper is also accessible to graduate students or practitioners familiar with LLM agents who are looking for a concrete architecture for multi-role, closed-loop generation systems. It assumes some familiarity with benchmark scoring conventions (pass rates, rubric scores on 0–100 and 1–10 scales) but does not require deep technical background in reinforcement learning or graphics.

Authors’ abstract

Recent game design agents have made substantial progress in generating playable games. However, program correctness does not ensure an enjoyable experience for players. We present Recursive Game Creator, an experience-oriented harness to advance agentic game development from rough game prototypes into entertaining games. Recursive Game Creator organizes recursive development around four components: Designer, Builder, Player, and Reviewer. The Designer translates user instructions and Reviewer's feedback into detailed plans. The Builder turns these plans into candidate games. The coding-native Player creates and executes reusable policies through programmatic interfaces to efficiently collect diverse gameplay trajectories, mitigating evaluation bias caused by slow GUI-based collection. The Reviewer uses carefully designed trajectory-based metrics to induce player preferences, integrating with visual evidence and explicit textual preferences to evaluate games against game-specific criteria. Finally, the Reviewer accepts the better version and provides improvement reviews for the next round, closing the recursive loop. Our method achieves state-of-the-art overall performance of 77.89 on GameCraft-Bench. On GameASG-Bench, it achieves a strict task success rate of 53.2%, a 34.1% improvement over the same-model baseline, and the highest mean runtime-check pass rate at 93.4% among compared methods. A user study shows longer playtime and higher ratings. Code is coming soon.