Procedura: Agentic 3D Modeling with Procedural Control

AI summary

Overview

• Research area: Computer vision and computer graphics, specifically 3D asset generation driven by large language models (LLM) code agents, with connections to CAD assembly modeling, robotics simulation, and physically based rendering.
• Technical level: Advanced. The paper assumes familiarity with constructive solid geometry, mesh connectivity analysis, joint kinematics, and LLM agent loops, though the core idea is stated simply enough for a motivated non-specialist.
• Scope: The paper introduces Procedura, a framework in which a frozen LLM writes an object as an editable procedural assembly program — named parts joined by typed, machine-checkable mates — rather than predicting a mesh, and evaluates it on two benchmarks against native 3D generators, prior 3D-code agents, and single-shot LLM prompting.

What This Paper Is About

Native 3D generators can recover impressive mesh geometry from a single image, but the resulting meshes are dense and soft where a machined object needs sharp edges, carry no part decomposition, and expose no parameter a user could edit. The authors test an alternative paradigm — "3D shape as code" — in which an LLM writes the program that builds the object instead of the object itself. Their goal is a text-prompt-to-3D pipeline that delivers three things at once: high-precision geometry, a meaningful part decomposition, and editability, without any 3D-specific training.

Key Contributions

1. A 3D-shape-as-code modeling agent. Procedura is an agentic framework that leverages and scales the coding ability of a frozen LLM for 3D modeling, producing detailed, many-part 3D shapes from a text prompt with no 3D training. Planning, building, and the critic use one frozen deep-reasoning LLM, and every other component is deterministic.

2. Procedural assembly with typed mates. An object is represented as an ordered set of named parametric CSG part modules plus a set of typed mates — nine static mating features (bolt-pattern, peg-socket, seat-face, flange, tab-slot, press-fit, lip-rabbet, snap-tab, key) and three kinematic ones (revolute, prismatic, spherical). Placement is solved from the mated frames rather than guessed, every commit is verified by a compile gate, a mate gate, and a connectivity gate, and parts stay editable by name.

3. The same graph completes a production asset. The mate network doubles as the material assignment structure (per-part PBR materials plus sub-part colour blocks) and as the kinematic graph, yielding an OpenUSD articulation and a URDF that is validated headless in Isaac Sim.

4. A benchmark and state-of-the-art results. The authors release MechBench-36, a frozen set of 36 deliberately many-part objects (rovers, mechs, cranes, excavators, engines, landing gear), and report that Procedura outperforms state-of-the-art native 3D generators and all prior 3D-code agents on judged quality on both MechBench-36 and the assembly task of P3D-Bench, with the sharpest edges and the only editable, part-structured output.

Main Findings

• Top overall score on MechBench-36. Procedura with Gemini 3.7 Flash takes the top judge composite at 0.828, leading geometry (0.799), aesthetics (0.827), and CLIP (82.67), and ships a mesh for every case. TRELLIS.2 follows at 0.810; Adam CAD at 0.799 and the GPT-5.6-sol arm of Procedura at 0.797 are effectively tied.

• The judge axes separate methods unequally. Semantic fidelity spans only 0.775 to 0.861 across methods that work at all; Adam CAD and TRELLIS.2 share its top at 0.861, just above Procedura's 0.858. Geometry and aesthetics separate the field far more, spanning 0.583 to 0.799 and 0.105 to 0.827, and Procedura tops both.

• Native generators lose on edge structure. The strongest native method, TRELLIS.2, trails by 5.2 CLIP points, and Direct3D-S2 and Hunyuan3D fall behind on aesthetics by 0.27 and 0.24. Procedura's programs carry 158.1 units of sharp edge above a 60° dihedral with Gemini 3.7 Flash and 185.2 with GPT-5.6-sol — 2.6× and 3.1× the next code-emitting method — at 95th-percentile dihedrals of 98.0° and 105.3°. Native generators sit at 33–73°; TRELLIS.2 posts the only comparable sharp-edge length at 134.0, but its 73.1° dihedral shows those creases are far shallower.

• Domain-tuned CAD coders collapse on many-part objects. CAD-Coder scores 0.210 aesthetics and cadrille 0.105, shipping no mesh on 1 and 5 cases respectively, which the authors attribute to a model trained to emit one sketch-extrude body being unable to express an assembly.

• The strongest prior agent remains behind. Adam CAD on GPT-5.6-sol reaches 0.759 aesthetics and 81.26 CLIP on MechBench-36, versus 0.827 and 82.67 for Procedura with Gemini 3.7 Flash.

• Paired gains over the base model. Paired per object over the whole benchmark, the pipeline improves aesthetics by +0.105 on Gemini 3.7 Flash (20 wins, 7 losses, sign test p = 0.019). On GPT-5.6-sol the mean gain is +0.059 but the split is 15 wins to 11 losses (p = 0.56), so the authors do not claim significance there. CLIP improves by +1.80 (p = 0.011) and +0.85 respectively.

• P3D-Bench assembly results. On the 203 text-and-image cases, Procedura with Gemini 3.7 Flash scores semantic 0.666, geometry 0.490, aesthetics 0.616, and judge overall 0.590 ± 0.009; Procedura with GPT-5.6-sol reaches 0.575 ± 0.010. The next best entries are single-shot Gemini 3.7 Flash at 0.566 ± 0.016 and single-shot GPT-5.6-sol at 0.563 ± 0.015, with the remaining baselines descending from GPT-5.5 at 0.524 ± 0.012 to MiMo Omni at 0.305 ± 0.012.

• Connectivity measured by span, not volume. A thin detached panel has near-zero enclosed volume yet a large span. The authors report a worst case from their audit of a pre-gate pipeline of 70% span and 1.9% volume, which is why the gate thresholds on span with τ = 0.01.

• Gating diagnoses changes agent behaviour. Without the diagnose-before-edit latch, an ungated variant issued 15 edits against 2 diagnoses; the gated loop holds the diagnose-to-edit ratio at approximately 1:1.

• Asset completion figures shown in the paper. The material stage extracts a compact library of typically one to two dozen PBR entries (25 entries for the assault buggy, covering 41 named parts) in four one-shot vision calls with no agent loop. The rover articulation example has 17 links and 16 joints, is validated in Isaac Sim, and is tested with a nine-wheel drive.

• Not reported in the provided content. The paper's section 4.4 (stage ablations) and section 4.5 (production-asset evaluation) results are not present in the supplied text, which ends at the start of the P3D-Bench discussion. Quantitative ablation numbers and any asset-level evaluation figures beyond those cited above are therefore not available here.

Methodology in Plain English

The authors do not train a 3D model. Instead, they wrap a frozen LLM in a harness that writes an OpenSCAD program, compiles it with the OpenSCAD engine, and renders the result.

• Representation. An object is a procedural assembly: an ordered list of named parametric CSG modules, plus mates that state how each new part joins an already-placed partner. Because faces are planes and bores are cylinders, edges stay sharp at any resolution. Because the module list is the decomposition, no post-hoc segmentation model is needed. Because every dimension is a named parameter bound to a named part, editing means changing a parameter.
• Planning. The agent synthesizes a single reference view from the prompt, then a vision call decomposes the object into an ordered list of parts, each with a snake_case name, a coarse detail level, a short description, and its interface declarations (the mates binding it to earlier parts). The list is a topological order of the assembly graph, so every part seats on geometry that already exists. A second vision call may review the plan but may only add a part or sharpen a description or mate — it can never merge, remove, rename, or reorder, and this is enforced in code.
• Building one part per call. Each LLM call adds exactly one module in a five-block format (new parameters, helper modules, part body, published interface frames, placement intent). A single call that must fix every part, count, and placement at once tends to leave misplaced sub-assemblies and floaters; growing the program one module at a time makes each call a small problem.
• Solved placement instead of guessed transforms. The model authors the part's geometry and its mate frames but not the assembly transform. For a static mate joining the new part's frame F_j to the placed partner frame F_i, the placement is the rigid transform T_j = F_i · Δ(φ) · F_j⁻¹, where Δ(φ) is the signed fit offset along the mate axis. Solving this in code removes what the authors call the largest single source of drift in prior 3D-code systems.
• Three grounded context sources per call. The assembly reference (reference image, prompt, plan slice), the built history (the full already-compiling program, passed as text so exact parameters and frames are read rather than guessed), and 3D feedback (a parts-colour render from up to twelve viewpoints — six orthographic faces and six isometric corners).
• Deterministic verification. A spliced part must clear a compile gate, a mate gate that measures registration area a(c) ≥ τ_a d² and penetration δ(c) ≤ δ_max(φ) on the compiled mesh, and a delta-aware connectivity gate that rejects a part only when it increases the count of visible floaters over the running baseline. Each part gets up to three quality attempts; a part that exhausts them is committed with a warning rather than dropped, since completion matters more than a defect the refine loop can still remove, while a part that never compiles is skipped.
• Decoupled-critic refinement. A separate single-turn vision call — same underlying model, its own system prompt and context, never a turn of the coding agent — sees the reference, the rendered views, the parts-colour legend, and the current program, and returns a summary plus a prioritized list of issues tagged HIGH, MED, or LOW, each naming the responsible module and a one-sentence fix direction. A successful diagnosis sets a one-shot latch that exactly one subsequent edit consumes; edit tools refuse without the latch, and the critic refuses if the rendered views are stale. The agent chooses 1 to 20 views from a fixed catalog, and the loop ends with ok when the critic reports no HIGH issue, or give_up when several edits fail to shrink the issue list.
• Connectivity as a hard constraint. The shipped mesh is welded and partitioned into connected components by union-find over shared edges. A component with non-zero span fraction ρ_i ≥ τ (τ = 0.01) that is not the largest body counts as a visible floater, and finish(ok) is refused while any remain. If the agent edited after its last compile, the committed program is recompiled so the gate grades the mesh the user actually receives.
• Evaluation protocol. Every shipped model is re-rendered from one fixed view set under one grey-clay rig, assigned an opaque hash, and judged blind by a vision LLM on semantic fidelity, geometric quality, and aesthetics. Alongside the judge, the paper reports CLIP between prompt and shared renders and two edge measurements on the shipped mesh: total sharp-edge length above a 60° dihedral and the 95th-percentile dihedral.

Why This Matters

Impact on research. The paper reframes 3D generation as program synthesis rather than surface prediction, and argues that the missing ingredients are structure (named parts, typed relations) and verification (solved placement, compile and mate gates) rather than raw model capacity. It also argues against self-refinement loops in which a model critiques its own output, showing that a decoupled reviewer is what allows a diagnosis to become a persisted, first-class artifact. The released MechBench-36 targets the many-part regime — part counts, placement, and connectivity — that existing image-to-mesh benchmarks stress less.

Real-world applications.

• Game and simulator content pipelines, where assets must be addressable by part and re-parameterizable rather than shipped as one opaque surface.
• Robotics and physics simulation, since the same mate graph exports an OpenUSD articulation and a URDF that drops into a simulator or robotics stack.
• CAD-adjacent design workflows, where edits are parameter changes on named parts rather than direct surface sculpting, opening the process to non-expert users.
• Look development and rendering, because per-part PBR materials and a Blender-Cycles studio render come directly from the reference image.

Industry relevance. The claim that Procedura is the only evaluated method whose output is an editable, part-structured program speaks to the gap between a photoreal mesh and a usable production asset. The paper also positions itself against native generators, which need large 3D training datasets that the authors describe as hard to collect, whereas Procedura requires no 3D training and uses off-the-shelf external binaries beyond the compiler (Blender for renders, headless Isaac Sim for physical validation). The sharp-edge measurements matter where a machined or engineered look is the requirement: compiled CSG edges are exact intersections of analytic surfaces, while regressed surfaces stay soft.

Future Directions

• Close the asset-level evaluation. The paper lists production-asset delivery as an experimental question (section 4.5) but the supplied text does not report those results, so how the material, sub-part colour-block, and articulation stages fare quantitatively remains open.
• Reduce the reliance on vision-LLM judgment. Every quality claim here rests on a vision LLM judge under a blind hashing protocol, and the authors themselves note that on MechBench-36 no ground truth exists for a prompt. Sharper automatic metrics for assembly correctness would strengthen the field.
• Extend beyond the current mate vocabulary and object scope. The representation is limited to nine static mating features, three kinematic types, four fit classes, and a fixed set of detail levels, and the hard-surface benchmark is deliberately machinery-focused; whether the paradigm holds for organic or deformable objects is not addressed.
• Harden the failure modes. Parts that exhaust three quality attempts are committed with a warning rather than dropped, a part that never compiles is skipped, and a part with several static mates is placed by the first and only checked against the rest. Whether these compromises accumulate in larger assemblies is an open question.

Target Audience

This paper is most useful to 3D generation and graphics researchers, LLM agent researchers interested in tool use, verification, and critique separation, and practitioners building asset pipelines for games, simulation, or robotics who need editable, part-structured models rather than dense meshes. Readers working on CAD program synthesis, LLM-authored 3D code, or sim-ready articulation will find the mate-graph representation and the deterministic verification stack to be the distinctive technical content. A working knowledge of constructive solid geometry, mesh connectivity, and kinematics is assumed; the framing and the plain-language explanations of each pipeline stage make the argument legible to graduate students entering the area.

Authors’ abstract

Native 3D generators now recover impressive mesh geometry from a single image. However, a dense mesh stays soft where a machined object should be sharp, it carries no part decomposition, and it exposes no parameter a user could edit. To address this, we explore the paradigm of 3D shape as code, leveraging and scaling the coding ability of an LLM for 3D modeling. We introduce Procedura, a novel 3D modeling agent framework that writes an object as a procedural assembly, a parametric program whose named parts are joined by typed, machine-checkable mates. From a text prompt, the agent plans the object as an assembly graph and writes the program part by part, solving each placement from the mated frames rather than guessing it, and admitting a part only once compile, mate, and connectivity checks pass. A decoupled vision critic then refines the assembly one diagnosed fix at a time. Moreover, the same graph carries per-part materials and a simulator-validated articulation. We evaluate on P3D-Bench under its assembly judge, and with the same judge on MechBench-36, our hard-surface benchmark. On both, Procedura outperforms state-of-the-art native 3D generators and every prior 3D-code agent on judged quality, produces the sharpest edges of any method we evaluate, and is the only one whose output is an editable, part-structured program.