CADKnitter: Compositional CAD Generation from Text and Geometry Guidance

AI summary

Overview

Research area: Computer Vision / 3D generative modeling, specifically computer-aided design (CAD) generation using Boundary-representation (B-rep) models and diffusion models.

Technical level: Advanced. The paper assumes familiarity with diffusion models, guided sampling, B-rep topology, Chamfer distance, and optimal transport (Fused Gromov–Wasserstein distance).

Scope: The paper introduces the task of compositional CAD generation, a dataset (KnitCAD) supporting it, and a diffusion-based framework (CADKnitter) that generates a complementary CAD part fitting a given CAD model while following a text prompt.

What This Paper Is About

Existing CAD generation methods focus on producing a single part in isolation, but real CAD work involves assemblies of multiple parts that must satisfy strict geometric and semantic constraints. This paper defines "compositional CAD generation": given a text prompt and an existing conditioned CAD model, generate a new complementary CAD part that physically fits the existing one and matches the design intent described in the text. The authors build both a dataset with the necessary annotations and a generation method that explicitly enforces geometric compatibility during sampling.

Key Contributions

1. KnitCAD dataset: A large-scale dataset for compositional CAD generation containing 313,308 samples (described in the abstract as over 310,000), derived from 156,654 unique assembled pairs across 172,265 distinct B-rep models, with textual prompts, assembly metadata, and automatically annotated contact faces. Table 1 lists the dataset as 157K samples in B-rep representation and identifies it as the only compared dataset with object pair labels, contact face labels, and text all present.

2. A compositional CAD generation network: A text- and geometry-conditioned diffusion model built on BrepGen's four-stage pipeline, modified in the first stage (face bounding boxes) to accept both semantic conditioning from text and geometric conditioning from the conditioned CAD model, with learnable embeddings injected at contact face positions.

3. A geometric-guiding cue with a scoring function: A derivative-free guidance mechanism using a scoring function that combines a geometry alignment term (Chamfer distance, D_geo) and a semantic preservation term (Fused Gromov–Wasserstein distance, D_sem), plus a guiding-cue predictor that optimizes contact faces via positional and shape costs.

4. Experimental validation: Demonstrated improvement over PivotMesh, BrepGen, and MatchMaker on CD, PR, IV, and VR metrics, plus human preference studies and cross-set generalization from AutoMate to Fusion 360 Joint subsets.

Main Findings

• Main benchmark results: On KnitCAD, CADKnitter achieves CD 83.38, PR 0.23, IV 7.59, and VR 0.44. BrepGen reports CD 116.82, PR 0.43, IV 16.71, VR 0.47; MatchMaker reports CD 102.15, PR 0.42, IV 18.95, VR 0.29; PivotMesh reports CD 137.70 only (it generates meshes, not CAD). The ablated version without guidance reports CD 88.69, PR 0.24, IV 9.42, VR 0.45.

• Guidance trade-off: Applying guidance lowers Valid Ratio (VR) by approximately 2% while improving Intersection Volume Percentage (IV) by approximately 19%, which the authors describe as substantially better geometric constraint satisfaction with a modest reduction in plausibility.

• Cross-set generalization: Trained only on the AutoMate subset and tested on the Fusion 360 Joint subset, CADKnitter reports CD 82.00, PR 0.26, IV 13.90, VR 0.48, versus BrepGen (CD 107.62, PR 0.40, IV 22.86, VR 0.48), MatchMaker (CD 106.44, PR 0.51, IV 23.40, VR 0.10), and PivotMesh (CD 142.56 only).

• Human evaluation: With 20 users aged 18–27 (designers, engineers, mechanics, students) and 1,000 comparisons, users preferred CADKnitter over PivotMesh with win rates of 0.84 (semantics) and 0.86 (geometric compatibility), over BrepGen with 0.56 and 0.70, and over MatchMaker with 0.49 and 0.67. Tie rates against MatchMaker were the highest (0.37 semantics, 0.19 geometric compatibility), which the authors attribute to MatchMaker's competitive semantic alignment.

• Scoring function ablation: Removing the semantic term (D_sem) degrades results to CD 87.50, PR 0.26, IV 8.52, VR 0.43, compared with the full scoring function at CD 83.38, PR 0.23, IV 7.59, VR 0.44.

• Predictor ablation: Removing the shape cost (C_shape) degrades to CD 88.81, PR 0.25, IV 8.72, VR 0.44; removing the positional cost (C_pos) degrades to CD 91.89, PR 0.24, IV 8.76, VR 0.43, versus CD 83.38, PR 0.23, IV 7.59, VR 0.44 with the full predictor.

• Guiding-cue comparison across guidance mechanisms: With Zero-Order Search, the proposed cues give CD 83.38, PR 0.23, IV 7.59, VR 0.44 versus a naive conditional-contact-face cue at CD 87.12, PR 0.23, IV 7.65, VR 0.43. With SVDD, the proposed cues give CD 86.24, PR 0.23, IV 6.91, VR 0.45 versus naive at CD 87.21, PR 0.25, IV 7.16, VR 0.44. The naive approach achieves competitive geometric compatibility but compromises semantic alignment, shown by the larger CD gap.

• Statistical prior on assembled parts: On the AutoMate subset, 75.25% of assembled pairs have a relative boundary-edge length difference under 5%, and boundary edge lengths between mating parts show a Pearson correlation of 0.9480, clustering along the y=x diagonal. This motivates using boundary edge equality as the primary shape-alignment objective.

• Dataset statistics: KnitCAD builds on Fusion 360 Gallery Assembly – Joint Data (19,156 joint pairs from 23,029 B-rep models) and AutoMate (541,635 mate pairs from 376,362 B-rep models; Table 1 lists AutoMate at 542K samples).

Methodology in Plain English

The researchers treat a CAD part as a B-rep model, a collection of faces plus their topology. They start from BrepGen's four-stage pipeline (face bounding boxes, face points, edge bounding boxes, edge points) and reuse the last three stages unchanged. They replace the first stage with their own network that takes two conditions: a text prompt embedded with BERT and the conditioned CAD model. Both conditions are fused with the noisy sample through cross-attention in a Transformer denoiser.

Because a stochastic diffusion model cannot be relied on to satisfy hard geometric constraints, they add a guidance step during sampling. They use derivative-free guidance (Zero-Order Search): at each guided step, they sample several candidate versions of the current intermediate result, score each candidate, and keep the best one. Guidance is applied at four late steps of the reverse process (timesteps 105, 85, 65, 45), where noise is low and decoded points are more reliable. To get points from intermediate results, they run the current face bounding boxes through BrepGen's pretrained face-point model, producing a 32×32 grid of points per face, without needing a fully valid B-rep.

The scoring function has two parts. The geometry term measures Chamfer distance between the candidate's contact faces and a predicted "guiding cue" (where the contact faces should ideally be). The semantic term uses Fused Gromov–Wasserstein distance to compare the candidate's global structure with the original intermediate sample, using face bounding box aspect ratios and normalized centers, solved with the Sinkhorn algorithm.

The guiding cue itself comes from an optimization: the contact faces of the intermediate sample are translated and reshaped to align with the conditioned model's contact faces. The cost combines a positional term (point-to-mesh distance) and a shape term built from boundary edge lengths and edge angles, based on the empirical finding that mating boundary edges are usually equal or proportional. Interior points follow their boundary points' displacements so faces deform coherently.

Training uses one NVIDIA A100 80GB GPU, AdamW, learning rate 5×10^-4, batch size 256, and 5,000 epochs, with the number of generated contact faces set to M=10. Models are initialized from public BrepGen weights pretrained on ABC. The condition model is normalized to be centered at the origin within [-3, 3] per axis, and the same transform is applied to the target.

Why This Matters

This work reframes CAD generation away from single isolated objects toward assemblies, which is closer to how CAD is actually used in engineering. It also supplies the dataset annotations (contact faces, assembly metadata, text) that prior CAD datasets lack, and shows that geometric hard constraints can be enforced in a non-differentiable CAD pipeline through derivative-free guidance.

Real-world applications:

• Mechanical and product design: generating mating parts like shafts that fit through existing holes, reducing manual iteration.
• Manufacturing and fabrication: outputs target nominal (3D printing-ready theoretical) size, so parts can be used directly or post-processed for fabrication.
• Robotics and simulation: assembling parts under stability and coupling constraints, which prior compositional shape work (such as MatchMaker) also targets for simulation.
• CAD software and design tooling: a component that takes an existing part plus a text description and proposes a fitting complementary part inside a design workflow.

Industry relevance: the work is published as a SIGGRAPH Asia 2026 Conference Paper, and the authors note practical demand for automated CAD generation in product design, mechanical engineering, and robotics. The project page is at https://cadknitter.github.io.

Future Directions

• Extending the statistical assumption beyond the studied sources: The paper validates the boundary-edge equality prior on the AutoMate subset and states that generalization is tested on an independent CAD source in Sec. 7, but the provided content does not report those results, leaving the scope of this assumption open.
• Scalability of the guidance mechanism: Figure 8 is titled "Scalability preservation" and the visible text cuts off at "scales with the number of candi…", so how guidance cost scales with the number of candidates is raised but not resolved in the provided content.
• Multi-part assemblies: The current formulation generates one complementary part given one conditioned model and a set of contact face indices; extending to larger assemblies with many interdependent parts is a natural next step.
• Improving the VR/IV trade-off: Since guidance lowers VR by approximately 2% while improving IV by approximately 19%, closing that validity gap remains an open problem.

Target Audience

Researchers and practitioners in 3D generative modeling, CAD and geometry processing, and computer graphics, particularly those already familiar with diffusion models and B-rep representations. It is also relevant to engineers and CAD tool developers interested in automated assembly-aware design, and to machine learning researchers working on constrained or guided sampling for non-differentiable outputs. Because it relies on B-rep terminology, optimal transport, and guided sampling, it is not beginner-friendly.

Authors’ abstract

Computer-aided design (CAD) defines 3D models as compact, precise, and editable representations, making it directly useful for several fields. Recently, CAD generation has been gaining more attention in both the research community and industry. Crafting CAD models has long been a painstaking and time-intensive task, demanding both precision and expertise from designers. Prior works have achieved early success in single-part CAD generation, which is not well-suited for real-world applications, as multiple parts need to be assembled under semantic constraints and geometric compatibility. In this paper, we propose CADKnitter, a compositional CAD generation framework with geometric-guiding cues to steer diffusion sampling. CADKnitter is able to generate a complementary CAD part that follows both the geometric constraints of the given CAD model and the semantic constraints of the desired design text prompt. We also curate a dataset, so-called KnitCAD, containing over 310,000 samples of CAD models, along with textual prompts and assembly metadata that provide semantic and geometric constraints. Intensive experiments demonstrate that our proposed method outperforms other state-of-the-art baselines by a clear margin. Our project page is available at https://cadknitter.github.io/.