StepCAD: Mesh-to-CAD Code Generation via LLM Policy and Geometry-Guided Search

AI summary

Overview

Research area: 3D shape analysis and computer-aided design (CAD) reconstruction, specifically recovering executable CAD programs from 3D meshes using generative modeling and search. Technical level: Advanced. Scope: The paper introduces StepCAD, a method that couples a state-conditioned CAD policy with geometry-guided tree search, along with a large-scale dataset of executable CAD programs, to improve mesh-to-CAD reconstruction accuracy and validity.

What This Paper Is About

Recovering executable CAD programs from 3D meshes is difficult because CAD construction is compositional: it involves discrete modeling choices and continuous parameters that interact. Many existing learning-based methods predict an entire program in one pass and focus mainly on sketch-extrude representations, which limits the variety of operations they can use and leaves little opportunity to fix geometric errors during reconstruction. StepCAD addresses this by generating CAD programs through a policy that conditions on both the target mesh and intermediate geometry, then refining the program with a search procedure guided by geometric overlap.

Key Contributions

1. A state-conditioned CAD policy that predicts construction actions based on both the target geometry and the intermediate geometry produced so far.
2. A geometry-guided search method, described as an IoU-guided tree search, that refines an initial CAD program through local edits.
3. ARCADE-1.5M, a large-scale dataset of 1.5 million executable CAD programs with diverse operations, sequences up to more than 150 counted operations, and 12.5 million intermediate state-action transitions.
4. Experimental results across multiple CAD reconstruction benchmarks showing state-of-the-art geometric reconstruction accuracy with consistently high validity, including up to 87.2% relative IoU improvement over the strongest evaluated baseline.

Main Findings

• State-of-the-art reconstruction accuracy: StepCAD achieves state-of-the-art geometric reconstruction accuracy across multiple CAD reconstruction benchmarks, according to the abstract.
• High validity: The method produces programs with consistently high validity, meaning the generated CAD programs are executable and well-formed.
• Large relative improvement: StepCAD yields up to 87.2% relative IoU improvement over the strongest evaluated baseline.
• Stronger on complex shapes: The largest gains are reported on complex shapes, where single-pass and sketch-extrude-focused approaches are most limited.
• Search enables correction: The IoU-guided tree search refines programs through local edits, providing opportunities to correct geometric errors during reconstruction rather than only after a single prediction.
• Beyond sketch-extrude: The approach supports diverse CAD operations rather than relying predominantly on sketch-extrude representations.

Methodology in Plain English

The researchers frame mesh-to-CAD reconstruction as a generative optimization problem. Instead of asking a model to output an entire CAD program in one shot, they use a policy that decides the next construction action while looking at both the input mesh and the geometry built so far. This intermediate-geometry conditioning lets the policy react to what it has already created. After the policy proposes a program, a search procedure explores local edits to the program, using geometric overlap (IoU) as a guide to decide which edits improve the reconstruction. To train and evaluate this approach, the authors assembled ARCADE-1.5M, a dataset of executable CAD programs that covers many operations, long operation sequences, and millions of intermediate state-action transitions. The abstract does not provide details on the policy architecture, search algorithm specifics, training procedure, or benchmark names.

Why This Matters

This work matters because it moves mesh-to-CAD reconstruction away from single-pass, limited-operation prediction toward a generate-and-refine process that can correct mistakes and handle more diverse CAD construction. It also contributes a large-scale dataset of executable CAD programs, which can support further research on learning CAD construction sequences. The abstract claims state-of-the-art geometric reconstruction accuracy and high validity, but it does not list specific benchmark names, per-benchmark scores, ablation studies, or runtime costs.

• Reverse engineering and part redesign: Converting scanned or legacy 3D meshes into editable CAD programs can help engineers modify existing parts rather than rebuild them from scratch.
• Manufacturing and CAD interoperability: Executable CAD programs are more useful than raw meshes for downstream manufacturing workflows, where operations and parameters matter.
• Digital twins and simulation: Recovering parametric CAD models from meshes can support simulation, analysis, and updating of digital representations of physical objects.
• 3D content creation and editing: Designers and artists could use mesh-to-CAD generation to turn scanned or generated 3D shapes into editable, programmatic models. Industry relevance: CAD reconstruction sits at the intersection of computer vision, graphics, and mechanical design. Improvements in accuracy, validity, and operation diversity could benefit software vendors, engineering teams, and manufacturing pipelines that need editable CAD from 3D scan data. The abstract does not claim deployment or production use.

Future Directions

• Extending the policy and search to even longer or more complex CAD operation sequences beyond the 150+ counted operations represented in ARCADE-1.5M.
• Improving the search procedure to scale to larger programs while keeping reconstruction accurate and valid.
• Testing generalization to CAD operations, design domains, or mesh qualities not covered by the dataset and benchmarks in this paper.
• Investigating how to make the generate-and-refine loop more efficient, since the abstract does not report search cost, inference time, or computational requirements.

Target Audience

This paper is most relevant to researchers and practitioners in computer vision, computer graphics, and CAD who work on 3D shape reconstruction, program synthesis, or generative models for design. It will also interest engineers and data scientists building tools that convert 3D meshes into editable CAD programs. Readers should have some familiarity with machine learning, 3D geometry, and CAD representations such as sketch-extrude operations.

Authors’ abstract

Recovering executable CAD programs from 3D meshes is challenging due to the compositional nature of CAD construction and the interaction between discrete modeling choices and continuous parameters. Many learning-based methods predict complete programs in a single pass and rely predominantly on sketch-extrude representations, limiting operation diversity and opportunities to correct geometric errors during reconstruction. We introduce StepCAD, a generative optimization approach that combines a state-conditioned CAD policy with geometry-guided search. Given an input mesh, the policy predicts construction actions conditioned on both target and intermediate geometry, and an IoU-guided tree search refines the resulting program through local edits. We also introduce ARCADE-1.5M, a large-scale dataset of 1.5M executable CAD programs spanning diverse operations, sequences with a maximum length of 150+ counted operations, and 12.5M intermediate state-action transitions. Experiments across multiple CAD reconstruction benchmarks show that StepCAD achieves state-of-the-art geometric reconstruction accuracy with consistently high validity, yielding up to 87.2% relative IoU improvement over the strongest evaluated baseline, with particularly large gains on complex shapes. Project page: https://ghadinehme.com/stepcad.github.io/