Clarify Before You Draw: Proactive Agents for Robust Text-to-CAD Generation

AI summary

Overview

Research area: Machine learning for program synthesis, specifically text-to-CAD (Computer-Aided Design) generation using large language model agents, with a focus on proactive dialogue and clarification.

Technical level: Intermediate to Advanced. The paper assumes familiarity with supervised fine-tuning, Chamfer distance, LLM agent pipelines, and parametric CAD representations, though the core idea is conceptually simple to describe.

Scope in one sentence: The paper introduces ProCAD, a two-agent system (a proactive clarifying agent plus a CadQuery coding agent) that detects and resolves missing or conflicting geometric constraints in natural-language CAD prompts before generating code.

What This Paper Is About

Text-to-CAD systems let users describe a 3D part in natural language and receive an executable parametric CAD script, but real user descriptions are frequently incomplete or internally contradictory — dimensions are missing or constraints conflict. Existing fine-tuned models react by silently hallucinating the missing dimensions rather than flagging the problem. ProCAD addresses this by inserting a clarification stage: an agent audits the prompt, asks targeted questions only when needed, and produces a standardized, self-consistent specification that a second agent then compiles into CadQuery code.

Key Contributions

1. A proactive agentic framework for text-to-CadQuery. Instead of static one-shot generation or post-hoc refinement, the authors decompose the task into ambiguity detection and targeted questioning, specification correction, and code synthesis, modeled formally as a finite-horizon Markov decision process with reward R = -CD(y) - λ·C(h) that trades geometric fidelity against interaction cost.

2. ProCAD-coder, a coding agent fine-tuned on only 1.6K carefully curated samples, which matches or exceeds frontier closed models such as Claude 4.5 Sonnet on unambiguous prompts (invalidity ratio 0.9% versus about 13%, median Chamfer distance 6.6×10⁻⁵ versus 7.7×10⁻⁵).

3. A new text-to-CadQuery annotation pipeline that starts from raw CadQuery programs and generates grounded natural-language descriptions using GPT-5-mini conditioned jointly on multi-view renderings and the code, with regex-free LLM-based code-leakage checks and a Geo/execution completeness check requiring Chamfer distance below 2×10⁻⁴. This yields a curated 10K-sample dataset.

4. ProCAD-clarifier, trained via agentic supervised fine-tuning on 6,063 synthetic clarification trajectories, which the authors report outperforms Claude Sonnet 4.5 and GPT-4o-mini on communication cost, corrected-prompt quality, and downstream geometry quality.

Main Findings

• Robustness gains on ambiguous prompts: The abstract reports that proactive clarification reduces mean Chamfer distance by 79.9% and lowers the invalidity ratio from 4.8% to 0.9% relative to frontier closed-source models including Claude Sonnet 4.5.
• Best two-agent configuration: In Table 4, pairing ProCAD-clarifier with ProCAD-coder on 2,469 ambiguous test samples yields the lowest mean CD (0.63), median CD (0.08), invalidity ratio (0.9%), and the highest Efficiency (0.9654) and Resolution (0.9341) scores, beating configurations where either agent is Claude Sonnet 4.5, GPT-4o-mini, or Qwen 2.5-7B-Instruct.
• Clarification helps even with a fixed frontier coder: Keeping Claude 4.5 Sonnet as the coding agent while swapping in ProCAD-clarifier drops mean CD from 3.10 to 0.85 and improves Efficiency from 0.8255 to 0.9665 compared with a Claude-Sonnet-as-clarifier configuration.
• Out-of-distribution simulator generalization: With user responses simulated by Claude 4.5 Haiku instead of the in-distribution GPT-5-mini, ProCAD-clarifier with ProCAD-coder still reaches mean CD 0.46, median CD 0.07, and invalidity ratio 0.9%, versus 3.14, 0.08, and 1.7% for the Claude Sonnet 4.5 clarifier.
• Human-centered evaluation agrees: On 100 examples with real annotators writing ambiguous prompts and CAD experts scoring, ProCAD-clarifier attains Efficiency 0.760, Resolution 0.787, mean CD 1.28, median CD 0.09, and IR 12.2%, outperforming Qwen 2.5-7B-Instruct, GPT-4o-mini, and Claude Sonnet 4.5 clarifiers on the interaction metrics.
• Dataset quality over quantity: Fine-tuning on 1.6K examples for 200 optimization steps (under 10 minutes on two H200 GPUs) achieves results competitive with prior work that uses more than 100K supervised examples, sometimes combined with reinforcement learning or refinement.
• Prompt brevity and judge preference: Compared to Text2CAD, the new prompts are much shorter (mean 147.8 vs 285.4 tokens; median 119.0 vs 228.0). Across 1,000 sampled pairs, LLM-judge win rates for clarity are 98.4% (Ours first) and 66.0% (Text2CAD first), and for human-likeness 100.0% (Ours first) and 96.5% (Text2CAD first).
• Base model matters in zero-shot settings: Qwen2.5-7B-Instruct shows an invalidity ratio of 82.9% on the new dataset and 86.9% on Text2CAD in zero-shot mode, with most failures attributed to CadQuery syntax errors rather than geometric reasoning.
• Text quality is a bottleneck: Replacing only the data pipeline while holding shapes and hyperparameters fixed changes results substantially (Ours IR 0.9%, Text-CAD 14.5%, JSON-Distill 5.3%, Claude 4.5 Sonnet 12.9% on 1K unambiguous prompts).
• Evaluation reliability: LLM-based and human evaluations of interaction quality agree on more than 96% of 100 randomly sampled test examples.

Methodology in Plain English

The authors break the workflow into three explicit stages rather than asking one model to do everything. First, a clarifying agent reads the user prompt and decides either to accept it as-is or to ask a batch of targeted questions. Second, after receiving answers, it emits a corrected, standardized specification. Third, a coding agent turns that specification into an executable CadQuery program.

To build the coding agent, the researchers worked backwards from code rather than forwards from text. They took an existing CadQuery corpus (about 17K samples from Rukhovich et al., derived from DeepCAD shapes), deduplicated the shapes, rendered each from four viewpoints, and prompted GPT-5-mini with both the images and the CadQuery program to write a natural-language description. Each description is checked for code leakage and for completeness (a separate model regenerates CadQuery from the description alone and its geometry must match within a Chamfer distance of 2×10⁻⁴). Failures are retried up to three times; the authors report over 80% of samples pass both checks with retries, and remaining cases go to CAD experts.

The coding agent, initialized from Qwen2.5-7B-Instruct, is fine-tuned with standard supervised next-token prediction on prompt–code pairs.

For the clarifying agent, the authors synthesize ambiguity by prompting GPT-5-mini to perturb verified specs in two ways: omitting key dimensions (under-specified) and assigning conflicting values (inconsistent). They keep a sample only if the original spec has CD below 2×10⁻⁴, the perturbed prompt exceeds 2×10⁻⁴, and the ratio between the two Chamfer distances is at least 10. Under the assumption that users can answer any clear question correctly, the optimal policy reduces to two rounds, so the clarifier is trained via agentic supervised fine-tuning on JSON-formatted targets covering acceptance, question generation, and final corrected specification. User responses during evaluation are simulated by GPT-5-mini, with Claude 4.5 Haiku used as an out-of-distribution simulator, plus a 100-example human study.

Why This Matters

Research impact. The paper reframes text-to-CAD from a single-pass translation problem into an interactive specification-repair problem, and argues that data-pipeline quality — not just model scale or training tricks — is a key bottleneck. Because it is, by the authors' claim, the first work to explicitly address ambiguity in text-to-CAD, it supplies a reusable annotation pipeline, a 10K dataset, a 6,063-sample clarification training set with full trajectories, and code released at a public GitHub repository.

Real-world applications.

• Engineering design assistants that flag missing dimensions before a part is modeled, rather than producing a plausible-looking but wrong solid.
• Rapid prototyping and 3D printing pipelines, where underspecified dimensions can cause physical parts to be unmanufacturable.
• Manufacturing and simulation workflows that require precise, editable parametric models rather than meshes.
• Educational or low-expertise CAD tools that lower the barrier to entry while still enforcing dimensional consistency.

Industry relevance. CAD is central to modern engineering and manufacturing, and the paper's demonstration that a 7B open-source model fine-tuned on 1.6K samples can rival Claude Sonnet 4.5 on valid code generation suggests a practical path to cheaper, self-hosted CAD copilots with bounded interaction overhead.

Future Directions

• Scaling and realism of interaction data. The authors note that modeling real user behavior for clarification is difficult and can vary substantially across annotators; their training data uses GPT-5-mini simulation only. Collecting large-scale real human clarification data is a natural next step.
• Richer ambiguity types. The perturbation pipeline covers only under-specified and inconsistent prompts; other specification failures in CAD practice are not addressed.
• Beyond the two-round assumption. The two-round optimum follows from assuming users answer any clear question correctly. Relaxing that assumption, for example with users who are uncertain or wrong, would require revisiting the multi-round formulation.
• Broader agent evaluation. The paper compares against single-model baselines and general-purpose LLMs, noting that Cadrille was trained on standard Text2CAD data and lacks ambiguity detection; additional clarification-aware baselines and benchmarks are left open.

Target Audience

Researchers and engineers working on LLM-based code generation, CAD/parametric modeling, or agentic dialogue systems; practitioners building design automation tools; and readers interested in how proactive clarification can make generative systems more reliable when user inputs are imperfect. The paper is most useful to those already comfortable with supervised fine-tuning and 3D geometric metrics such as Chamfer distance.

Authors’ abstract

Large language models have recently enabled text-to-CAD systems that synthesize parametric CAD programs (e.g., CadQuery) from natural-language prompts. In practice, however, geometric descriptions can be under-specified or internally inconsistent: critical dimensions may be missing and constraints may conflict. However, existing fine-tuned models tend to reactively follow the user instructions and hallucinate dimensions when the text is ambiguous. To address this, we propose a proactive agentic framework for text-to-CadQuery generation, named as ProCAD, that resolves specification issues before code synthesis. Our framework pairs a proactive clarifying agent, which audits the prompt and asks targeted clarification questions only when necessary to produce a self-consistent specification, with a CAD coding agent that translates the specification into an executable CadQuery program. We fine-tune the coding agent based on a curated high-quality text-to-CadQuery dataset and train the clarifying agent via agentic SFT on clarification trajectories. Experiments show that proactive clarification significantly improves robustness to ambiguous prompts while keeping interaction overhead low. ProCAD outperforms frontier closed-source models, including Claude Sonnet 4.5, reducing the mean Chamfer distance by 79.9% and lowering the invalidity ratio from 4.8% to 0.9%. Our code and datasets are made publicly available on https://github.com/BoYuanVisionary/Pro-CAD.