GOD: Govern, Observe, and Direct - A Real-Time Control Room for Agent Societies

AI summary

Overview

Research area: Multi-agent systems / LLM-driven generative agent simulation, specifically the tooling layer for operating, inspecting, and packaging simulated agent societies.

Technical level: Advanced. The paper presumes familiarity with generative-agent architectures (retrieval, reflection, planning loops), simulator runtimes, and replay/artifact packaging. The framing of the problem, however, is accessible to anyone who has run a simulation and struggled to inspect it.

Scope: GOD is a local-first browser control room that connects live operator commands (Ask and Intervene) over the bundled AgentSociety simulator and JiuwenClaw agent runtime to spatial replay evidence and portable experiment, map, and agent packs, evaluated on 15 completed run slots using Qwen-Plus.

What This Paper Is About

Generative-agent systems are easy to launch but hard to inspect: a run contains many agents, locations, messages, commands, and model calls, yet the operator usually receives either a finished replay or raw logs. That makes it difficult to ask why an agent moved, to test a small intervention, or to hand a run to another researcher. GOD's goal is to provide one browser workflow in which an operator can issue targeted questions or interventions and immediately inspect the resulting replay state, with everything governed by a shared command record and separated artifact contracts.

Key Contributions

1. An operator command loop. A browser control room connects mapped state, temporal controls (pause, step, scrub, replay), targeted natural-language questions, and next-step interventions in a single view, with agents remaining visible on the map throughout.

2. A shared command record. Live controls and replay views use the same command schema, so each Ask and Intervene action stays attached to the state transition it precedes. Every record stores target, prompt, result, simulation time, and step.

3. Separate artifact contracts. Runnable experiment packs, map packs, and agent packs are kept apart from replay history, credentials, and machine-local runtime state. Public packs include scenario, map, and profile data but exclude API keys, local configuration, runtime snapshots, and replay databases.

4. An integration layer rather than a new simulator. GOD does not introduce a new policy, planner, memory architecture, or base simulator; its contribution is the mapping between browser commands and execution/replay records, the authoring views around that layer, and the import/export contracts for portable packs.

Main Findings

• Completed run slots: 15 / 15 planned run slots completed. The scored set is one no-event baseline plus 14 intervention runs drawn from 10 scenario templates, four of which were run twice, all on the same PKU map with 22 profiles and identical initial locations, using Qwen-Plus through the DashScope endpoint.

• Event routing: 14 / 14 event commands avoided movement routing after a parser fix. The first pass exposed a routing collision because the parser trigger dao ("to/arrive") also occurred inside the Chinese compounds shou-dao ("receive") and dao-fang ("visit"); a regression test was added and the six affected runs were rerun from fresh directories. The authors state this 14/14 figure is a regression check on the corrected path, not a held-out estimate of routing generalization.

• Event-specific trace@2: 212 / 308 agent-runs showed a strong scenario term in the first two post-event frames or in the event-belief answer without an explicit denial. The staff-only variant left no such term in its two-step window even though its command response recorded targeted acceptance for the six selected agents. Pooled trace is 212/308 (68.8%); averaged within scenario first, it is 72.7%.

• Target destination recorded@1: 78 / 84 target-agent checks recorded the commanded destination as current or target location after one movement step. All six misses came from the gymnasium variant, where pathfinding reported the destination as unreachable.

• Non-target matches: 23 / 224 non-targets were at the commanded destination. This is reported separately because it can include agents who were already at that location, so it is not a causal side-effect measure.

• Replay-state string match: 169 / 182 state answers matched a saved location alias or action substring from the nearest replay frame.

• Event-boundary string match: 144 / 182 answers passed the stated term rule, combining 70/70 pre-event absence checks with 74/112 post-intervention presence checks. A separate audit found 0 / 140 pre-event event-term leakage.

• Negative controls: 0 / 392 answers mentioned an unsupported location (limited to the seven listed locations), and 9 / 252 post-answers missed a role-related profile term.

• Repeat stability: The mean pairwise final-location JSD across the four repeated scenario pairs is 0.011. Three pairs had JSD 0, while the diplomatic-visit pair had 0.045. The authors state that four pairs do not establish deterministic behavior, stability, or a timing effect.

• One operational failure: A discarded attempt of the delayed-notice run exceeded the 360-second request limit during a pre-event step; a fresh retry completed, and only completed-run markers entered the results.

• Software tests and artifact checks: The selected backend suite passed 82 tests covering replay export, package import/export, live endpoints, operator commands, and setup routing. The public site exposes two replay pages with 29 timeline frames, 10 and 22 agent entries, four completed operator commands, and eight release-backed replay downloads. The pack library exposes 10 experiment packs, 10 map packs, and 10 agent packs containing 141 profile entries, 104 location entries, and 173 interaction entries. A release-boundary audit scanned 635 public-data files and 141 public agent runtime configs for private-state files, local paths, and secret-like assignments and found no violations.

• No cross-system scores: Because the runner assumes GOD's command, replay, profile, and pack schemas, comparison with other systems would require adapters; no cross-system scores are reported. Table 1 compares only capabilities reported in the cited system descriptions (Generative Agents, AgentSociety, AutoGen Studio, AgentScope).

Methodology in Plain English

The authors built a control room, not a new simulation. The system pairs a React/Vite browser interface with a FastAPI backend wrapped around an existing AgentSociety simulator and the JiuwenClaw agent runtime (v0.1.11). The operator supplies the language-model endpoint used for local execution.

The workflow has a fixed shape. The operator picks or authors a scenario through a four-step setup wizard (model settings, scenario, agent review, launch), inspects the current world in a spatial replay interface, issues a natural-language command, advances the run, and preserves the resulting state. Two command types exist. Ask is a read-only interview: the backend calls the agent's external-question path with the current simulation time and requests a first-person answer grounded in the profile, current context, recent questions, and session state — it never modifies the environment. Intervene changes what the next step can see or do: movement commands call the map environment's pathfinder directly and expose path length, world-event commands call the environment event publisher (which can set the phase, add the event to later observations, and broadcast to the group mailbox), and other instructions are stored as pending interventions inserted into the next step prompt. Authoring tools (Map Studio, Agent Studio) edit profiles and map packages before local publication.

For evaluation, the researchers ran a benchmark runner that calls the same Ask, Intervene, and Run Step endpoints the browser uses, then reads the resulting SQLite replays and command transcripts. Each non-baseline run follows one script: ask five agents of different roles (student, teaching assistant, librarian, reporter, coordinator) whether they know of a public event and where they are; inject the event; advance two steps and re-ask about state, event belief, and one social relation; target a movement intervention at the scenario destination; advance one more step; ask three target agents why they are at their current location. Event metrics are therefore measured at @2, while target-destination recording is measured at @1. Scenarios cover public warnings (a volcano warning, an earthquake warning, a school power outage, a traffic blockade), a diplomatic visit, a rumor that an event was canceled, a public lecture, and three controlled variants that change exactly one variable: notification target (staff-only diplomatic notice), notification time (two ordinary steps before the diplomatic notice), and destination (public lecture moved to the gymnasium). Scoring is deterministic string checking — location aliases, action substrings, and a fixed list of strong scenario terms — with the authors explicitly stating these labels do not establish semantic grounding, hallucination, or persona quality. The demonstration uses the fictional GOD Town scenario with 10 residents, 10 semantic locations, and 65 location-scoped interactions, advancing in 30-minute in-world ticks.

Why This Matters

Impact on research. Most work on generative agents focuses on agent quality. GOD focuses on the operator's ability to interrogate a run, intervene mid-run, and hand the artifacts to someone else. The paper's clearest methodological point is that routing evidence and behavioral evidence are different things: the staff-only notice produced no strong event term in its two-step window while its command response recorded targeted acceptance, which is why the authors keep those two measures separate. The paper also states plainly what its string checks cannot claim, and reports pooled and scenario-macro numbers side by side.

Real-world applications.

• Controlled scenario comparison for NLP and social-simulation researchers, holding maps and profiles fixed while varying an event, its recipients, its timing, or a destination.
• Classroom walkthroughs, where an instructor can pause, ask an agent a question, inject an event, and immediately show how the next frames change.
• Design and prototyping of interactive worlds, using Map Studio and Agent Studio to draft and validate locations and residents before committing to a run.
• Sharing inspectable simulation artifacts: hosted read-only replays that require no credentials, plus downloadable packs another researcher can edit and rerun locally.

Industry relevance. GOD is released under Apache-2.0, integrates AgentSociety and JiuwenClaw subtrees, and derives its The Ville map assets from the Generative Agents project with upstream licenses and attributions retained. It is positioned against developer-facing tools such as AutoGen Studio and AgentScope; the distinctive claim is that commands are recorded alongside spatial replay state and that scenario data is exported on a separate contract from runtime state. The release-boundary audit (635 public-data files, 141 public agent runtime configs, no violations found) reflects a practical concern for anyone shipping simulation artifacts publicly.

Future Directions

• Broader model coverage. All scored runs used Qwen-Plus with the provider's default temperature held fixed, so results may differ with other models; the authors note the findings describe this one configuration rather than model-independent agent behavior.

• Stronger and larger-scale evaluation. Four repeated scenario pairs produced a mean pairwise location JSD of 0.011, with three pairs at 0 and the diplomatic-visit pair at 0.045 — a pilot comparison, not a demonstration of stability. Similarly, 0/392 unsupported-location mentions applies only to the seven listed locations.

• From string checks to semantic measures. Replay-state and event-boundary matching are lexical checks over fixed simulated profiles; the authors state they do not establish semantic grounding, hallucination, or persona quality. The natural next step is measures that test meaning rather than substring presence.

• Operator usability and destination execution. Operator usability was not evaluated, and target destination recorded@1 checks recorded state after one movement step rather than completed trajectory execution or arrival. Both are open questions the paper leaves explicitly unanswered.

Target Audience

NLP researchers and agent-system builders who need to inspect, intervene in, and share simulation runs without credentials or hand-edited environment files; educators running classroom walkthroughs of agent societies; designers prototyping interactive worlds; and engineers building developer tooling around multi-agent runtimes who want a concrete example of aligning a command schema across live controls and replay evidence.

Availability: GOD is released under Apache-2.0 at https://github.com/XiaoLuoLYG/GOD, with a public site at https://xiaoluolyg.github.io/GOD/ hosting replays and downloadable experiment, map, and agent packs. Hosted replays require no credentials; new runs require a local model endpoint. The GOD Town scenario's setting and visual assets are adapted from Generative Agents, and third-party subtrees and derived visual assets retain their upstream licenses and attributions.

Authors’ abstract

Generative-agent systems are easier to start than to inspect. A run can contain many agents, locations, messages, commands, and model calls, yet the operator often gets either a finished replay or raw logs. That makes it hard to ask why an agent moved, test a small intervention, or package a run for another researcher. GOD is a local-first control room for agent societies. From the same browser workflow, an operator can issue targeted questions or interventions and inspect the resulting replay state. The system combines a setup wizard, Agent Studio, Map Studio, a spatial replay interface, Ask and Intervene commands, and portable experiment, map, and agent packs. Its technical contribution is the command and artifact loop: live controls and replay evidence share the same operator command model, while package contracts separate scenario, map, and profile data from local runtime state. The public release includes hosted Smallville-style and PKU replays, the open-source repository, and downloadable packs. We evaluate this path on 15 completed run slots. Across the 14 intervention runs, 78 of 84 target-agent checks recorded the commanded destination, and 169 of 182 state answers matched a saved location or action string.