Compositional Machine Design as Program Synthesis with LLMs

AI summary

Overview

• Research area: Artificial Intelligence — LLM agents, program synthesis, and physically grounded design (generative mechanical reasoning under physics simulation).
• Technical level: Advanced (assumes familiarity with LLM agents, reinforcement learning, and program synthesis; the core ideas are explained here in plain language).
• Scope: The paper introduces compositional machine design as a form of program synthesis in which LLMs write construction programs for machines whose correctness is judged by simulated physical behavior, and introduces BesiegeField, a testbed built on the machine-building game Besiege, along with benchmarks of LLM agents, reinforcement-learning experiments, and a short human reference study.

What This Paper Is About

Most program-synthesis benchmarks test generated code in symbolic or digital environments, where correctness is decided by parsing, test cases, or execution output. This paper asks whether LLMs can instead write programs whose "correctness" is decided by physics: a machine is written as a construction program that composes standardized parts (blocks, wheels, hinges, springs, propellers, cogs, and so on), and the resulting machine is dropped into a simulator to see whether it moves, throws, lifts, or flies as requested. The goal is to isolate the core challenge of translating a textual functional demand into a spatially precise composition that actually works under gravity, contact, and actuation.

Key Contributions

1. A new formulation: The paper formulates compositional machine design as an LLM-agent problem, where models synthesize construction programs for machines evaluated by physical simulation rather than by symbolic tests.
2. A testbed: It introduces BesiegeField, an interactive environment adapted from Besiege, with standardized mechanical components, a construction DSL based on a construction-tree representation, scripted physical scenes, rollout state feedback, and simulation-derived rewards.
3. A benchmark: It benchmarks LLM agents across seven machine-design task families (Movement, Throwing, Delivery, Jumping, Lifting, Flying, and Gearing), comparing single-agent generation, iterative editing, and hierarchical design workflows.
4. Analysis of representation, feedback, and learning: It analyzes the roles of construction representation, simulation feedback, and RL fine-tuning (including effects on mechanism-level exploration), and adds a short-exposure human reference study comparing human and LLM design behavior.

Main Findings

• LLMs can synthesize nontrivial machine programs: Strong models recover task-relevant structures and sometimes reach nonzero rollout performance. In Movement, successful designs often include stable bases, support structures, and powered wheels in useful orientations; in Throwing, agents sometimes produce launcher-like assemblies with projectile holders, pivots, supports, and counterweights.
• Failures are mainly structural and physical, not formatting errors: Common patterns include incorrect part orientations, wrong parent attachments, missing supports, instruction-following failures, and flawed mechanism choices. Some machines have plausible components but lack the geometry to transfer motion or stay stable, and some show a mismatch between the design chain-of-thought and the emitted construction tree.
• Simulation feedback and edit history help refinement: Feedback improves performance over one-shot generation, especially when it exposes editable failures such as motion in the wrong direction, failed projectile release, orientation drift, or broken components. Edit history reduces repeated invalid attempts and helps preserve useful partial structure. Scalar reward alone is less useful than state feedback, because it ranks candidates but does not reveal which attachment, orientation, or support caused a failure.
• Hierarchy helps when the high-level plan is reliable: Hierarchical construction improves mean performance when the meta-designer produces a mechanically meaningful decomposition (as in the stronger Gemini 2.5 Pro setting) and tends to reduce variance. For weaker models, a flawed high-level blueprint can cause downstream agents to refine the wrong mechanism rather than replace it.
• Representation matters substantially: The construction-tree representation outperforms coordinate-based (global position-based) generation. In the reported ablation, Gemini 2.5 Pro reached machine validity of 5/8 and mean/max scores of 8.49/9.14 with the construction-tree representation, versus 5/8 machine validity and 4.96/12.85 with the global position-based representation; for that same representation, o3, Claude Opus 4, and Llama 4 Scout 17B 16E produced 0/8 parse-valid outputs.
• Parsed 3D summaries and design CoT help, but do not close the gap: Parsed 3D summaries improve performance in several settings, suggesting LLMs struggle to infer spatial relations from construction programs alone. When other models are conditioned on Gemini-generated CoT, their outputs align better with intended mechanisms and often improve, yet the construction bottleneck remains: knowing a catapult needs a lever or counterweight differs from compiling that plan into correct attachments, orientations, and actuation.
• RL improves the best machine found under a fixed generation budget: Fine-tuning Qwen2.5-14B-Instruct with GRPO on simulation-derived rewards most robustly improves best-sample discovery, while mean score and validity changes are task-dependent. Cold-starting matters: RL from the cold-start model outperforms RL from the base instruction model, and cold-start fine-tuning alone remains insufficient. Training curves show increasing validity, improving best score, and decreasing output token entropy.
• A Pass@k-style objective better matches design search: Because high-reward machines are sparse, the Pass@k-style objective discovers high-reward designs more effectively than the Pass@1-style estimator.
• RL mostly refines existing strategies: Qualitative evolution shows RL often improves local details (positions, supports, orientations, nearby components) while preserving the same high-level mechanism, rarely discovering qualitatively different strategies once training concentrates around a design family — consistent with rising reward and falling entropy.
• Humans and LLM agents search differently: In the short-exposure human study, tasks were difficult and participants did not immediately produce consistently high-performing machines, but they often alternated between planning, quick trial-and-error, and larger redesigns, whereas LLM agents tended to remain in local editing loops after proposing an initial mechanism.

Methodology in Plain English

The researchers adapted the game Besiege into an automated testbed. Instead of requiring LLMs to emit the game's native coordinate-based machine files — where small coordinate errors break connections — they exposed a compact construction DSL. A program starts from a root block and adds components one at a time through local attachment commands specifying the new component type and identifier, the parent component, the parent face, and the orientation; two-endpoint parts such as springs specify two parent attachments. Programs are serialized as JSON for the models, parsed, validated, and compiled into the native machine format for simulation.

Validity is checked at three levels: parse validity (can the output be decoded into the construction DSL), spatial validity (does the assembly satisfy attachment constraints and avoid severe spatial conflicts), and machine validity (both). Invalid machines get zero reward. Performance is reported as mean simulation score (reliability under a fixed generation budget) and maximum simulation score (whether at least one strong design was found).

In simulation, the machine is placed in a scripted scene and run under a fixed open-loop control protocol: it settles under gravity, then powered components activate on a fixed schedule. This isolates machine structure from controller search, so score differences reflect the generated program rather than a learned controller. The environment records positions, orientations, velocities, object displacement, projectile trajectories, and broken components, which feed both scoring and refinement feedback.

Three agent workflows are compared: (1) single-agent generation, where one LLM produces the whole program after a design chain-of-thought; (2) iterative editing, where a designer proposes a machine, an inspector critiques it structurally, and refiners propose multiple candidate edits that are evaluated by rollout, with the best branch selected by search; and (3) hierarchical construction, where a meta-designer decomposes the task into functional modules and designer agents build module by module with a beam-search-style procedure over valid partial machines, followed by the same inspection and refinement loop.

For learning, the authors built a cold-start dataset by collecting textual machine-function descriptions from Besiege player communities plus task-oriented prompts, prompting Gemini 2.5 Pro to generate design CoT and programs, and filtering for parse validity, machine validity, and functional behavior — yielding 9,982 valid machine–CoT pairs with no exact or paraphrased duplicates of the benchmark prompts. This data fine-tunes Qwen2.5-14B-Instruct. RL uses GRPO with reward R = 1_valid · P, where P is the task-specific performance score (travel distance for Movement; projectile distance and maximum height with constraints penalizing degenerate solutions like rolling or carrying the boulder for Throwing). Five baseline configurations were compared: base model, cold-start, RL from base, and cold-start + RL.

Why This Matters

• Research impact: The work defines a benchmark where machine-design programs are judged by what they physically do, not just whether they parse. It complements prior programmatic 3D-generation and CAD-script work, which mostly targets geometry, visual plausibility, or scene composition, and it differs from morphology–control co-design work by studying assembly from standardized parts under fixed controls. It is related to but distinct from concurrent work such as BuildArena (which defines Support, Transport, and Lift tasks under a shared agentic workflow), VLMgineer (robotic tool and action co-design), and Makatura et al. (2023) (one-shot mechanical structure generation that delegates parameter search to external optimization).
• Real-world applications:
  • Robot and mechanism prototyping, where a language description of a functional demand could seed candidate assemblies for simulation before any hardware is built.
  • Educational tools for teaching mechanics and design, using physics feedback as a learning signal.
  • Game and simulation content creation, where automated construction of functional machines or vehicles supports procedural generation.
  • Research on agentic engineering pipelines that need to convert language-level intents into executable, physics-tested structures.
• Industry relevance: The paper positions physics-simulated construction programs as a controlled intermediate step between text-to-CAD systems and real engineering. It explicitly cautions that its simplified physics and standardized components mean generated machines should not be read as safe or reliable engineering designs, and that real deployment would require objectives beyond task score, such as robustness, safety, fabrication cost, and ease of modification. The finding that reward maximization can narrow mechanism diversity is directly relevant to anyone deploying RL on design tasks.

Future Directions

1. Richer and CAD-compatible representations: Moving beyond the current rigid-body, fixed-scale block set toward multimodal interfaces, visual and geometric inspection, sketching, and embodied manipulation that human designers naturally use.
2. More realistic physics and engineering constraints: Incorporating material properties, manufacturing processes, tolerances, safety margins, cost, and fabrication-aware objectives, since the current benchmark omits all of these.
3. Joint morphology–control optimization: The paper deliberately excludes this by using a fixed open-loop control protocol and leaving joint optimization to future work; it notes that per-machine optimized controllers would create a coupled optimization problem and make scores incomparable across models.
4. Learning methods that preserve mechanism diversity: Treating structural diversity as part of the objective so models maintain a repertoire of functional mechanisms instead of collapsing onto the best-performing pattern, and further testing whether RL can discover qualitatively new mechanisms rather than refining existing templates.

Target Audience

Researchers and practitioners working on LLM agents, program synthesis, reinforcement learning with verifiable rewards, and generative design or engineering automation. It is also relevant to readers interested in embodied and physically grounded reasoning benchmarks, in text-to-CAD or morphology-design systems, and in evaluation methodology for agents whose outputs must survive contact with a physics simulator. The paper is written at an advanced level, but its central framing — machines as programs judged by simulation — is accessible to anyone familiar with LLM agent workflows.

Authors’ abstract

Large language models (LLMs) have shown strong abilities in writing and revising programs, yet many program-synthesis benchmarks still evaluate programs in symbolic or digital environments. We introduce compositional machine design, a physically grounded form of program synthesis where machines are written as programs that compose standardized parts, and success is determined by simulated physical behavior. To study this problem, we present BesiegeField, a testbed built on the machine-building game Besiege. In BesiegeField, LLM agents generate machine programs from textual functional demands, execute the resulting machines in simulation, and receive rewards and state feedback. We benchmark LLM agents across representative machine-design tasks under single-agent generation, iterative editing, and hierarchical workflows. Strong models recover task-relevant structures and sometimes achieve nontrivial physical performance, but often struggle with spatially precise assembly, mechanism-level planning, and translating feedback into useful structural edits. We further finetune Qwen2.5-14B, an open-source LLM, with reinforcement learning from simulation-derived rewards. We find that, under a fixed generation budget, RL improves the best machine discovered. We additionally evaluate human performance to provide a reference point for task difficulty. These results establish compositional machine design as a testbed for studying LLM agents that synthesize executable machine programs and improve them through physical feedback.