RTL-Interact: Training EDA-Interactive Models for Multi-Task RTL Design

AI summary

Overview

Research area: AI for electronic design automation (EDA) — applying large language models to register-transfer-level (RTL) hardware design, with a focus on post-training models to interact with EDA tools.

Technical level: Advanced. The paper assumes familiarity with LLM supervised fine-tuning, reinforcement learning (GRPO-style policy optimization), and hardware design flows (simulation, linting, synthesis).

Scope (one sentence): The paper presents RTL-Interact, a post-training framework that trains a single RTL-specialized model (8B and 32B) to generate Verilog, construct task-appropriate EDA checks, and revise designs from tool feedback, evaluated on CVDP, VerilogEval v2, and RTLLM v2.

What This Paper Is About

Writing usable RTL requires more than syntactically valid code: the design must compile, simulate correctly, and meet design objectives, and EDA tools are what tell you whether it does. Prior RTL-specialized models either use EDA tools only as training-time reward signals (not during inference) or split generation and repair into separate adapters or cooperating agents, adding complexity. This paper trains one model to generate RTL, decide which EDA checks to run, and revise its own design based on the returned feedback.

Key Contributions

1. A multi-task post-training framework that builds task-specific teacher demonstrations across three task types (generation, completion, optimization) and applies CoT-to-TIR staged supervision to train a single RTL-specialized model for both generation and feedback-guided revision — without separate adapters or cooperating models.
2. Pattern-Aware Policy Optimization (PAPO), which augments Group Relative Policy Optimization (GRPO) with correctness-gated rewards conditioned on the model's EDA-checking pattern, plus task grouping that preserves prescribed task-type proportions in each policy update.
3. A controlled study of supervision ordering at matched chat-template-token budgets (450M and 900M), showing that CoT-then-TIR staging outperforms TIR-only training, naive mixing, and the reverse order.
4. Empirical results where RTL-Interact-32B surpasses its tool-enabled teacher DeepSeek-V3.2 on CVDP (48.7% vs 46.7% weighted Pass@1; 68.9% vs 65.2% Pass@5), and both model sizes outperform the evaluated RTL-specialized baselines on CVDP, VerilogEval v2, and RTLLM v2.

Main Findings

• CVDP results: RTL-Interact-32B reaches 48.7% weighted Pass@1 and 68.9% Pass@5, exceeding the tool-enabled DeepSeek-V3.2 teacher by 2.0% and 3.7% respectively, and achieving the highest overall CVDP Pass@5 among compared models. RTL-Interact-8B reaches 44.9% Pass@1 and 63.2% Pass@5.
• VerilogEval v2 and RTLLM v2: On VerilogEval v2 code completion, the 32B model reaches 80.3% Pass@1 and 87.2% Pass@5, exceeding the teacher in both tool settings. On specification-to-RTL, the 32B model reaches 77.2%/86.5%. On RTLLM v2, the 8B and 32B models reach 77.6%/86.0% and 77.2%/88.0% Pass@1/Pass@5, exceeding all evaluated baselines in Pass@1.
• RL adds accuracy on top of SFT: Relative to SFT, RL yields CVDP Pass@1 gains of 5.4% (8B) and 6.2% (32B), with further Pass@1 gains on VerilogEval v2 and RTLLM v2.
• Tool-use efficiency improves alongside accuracy: Across the RL stage, the 8B model's mean tool calls per trajectory drop from 4.7 to 1.7 (a 63.8% reduction) while CVDP Pass@1 improves by 5.4%. Truncation rate falls from 33.7% to 15.2%, and mean response length drops 28.8% (8B) and 33.5% (32B).
• More thinking, fewer calls: The first thinking length grows to 2.8× and total thinking length to 1.5× their SFT values, indicating that fewer tool calls coexist with more explicit reasoning text.
• Staged supervision beats alternatives: At a matched 900M-token budget, CoT→TIR reaches 29.5% Pass@1 versus 28.5% for TIR-only, 28.2% for naive mixing, and 25.5% for TIR→CoT. CoT→TIR yields 97.4% tool use versus 0.0% for the reverse order.
• Both PAPO components matter (8B ablation): The full method scores 44.9%/63.2% overall Pass@1/Pass@5, ahead of the no-pattern-aware-reward variant (39.7%/59.9%) by 5.2%/3.3%, and ahead of the no-task-grouping variant (41.8%/60.3%) by 3.1%/2.9%, using an optimization-task share of α = 0.3.
• Generation and revision measured separately (100 CVDP tasks): Two-stage SFT raises direct generation and feedback-guided revision success from the base model's 17.0%/14.0% to 27.0%/26.0%, above CoT-only SFT's 23.0%/19.0%. PAPO further lifts revision success to 35.0% while keeping generation comparable at 26.0%.

Methodology in Plain English

The team started from an existing open-source dataset (87,321 CoT examples from CodeV-R1-dataset) and used DeepSeek-V3.2 as a teacher model. The teacher first scored specification–RTL pairs on four dimensions using 0–4 integer scores; only pairs marked Pass with a total above 10 entered a shared source pool.

From that pool they automatically built three kinds of tasks. In generation, the teacher writes RTL plus a testbench and revises using simulation output. In completion, an LLM masks the core logic of a reference design and the teacher must fill it back in, checked by equivalence checking against the unmasked original. In optimization, designs with lint warnings or high synthesis cost must be improved while preserving behavior, checked by both equivalence checking and the requested improvement. Each retained trajectory records the teacher's reasoning, tool requests, real tool outputs, revisions, and final RTL, yielding 13,480 trajectories (roughly 9K generation, 3K completion, 1.5K optimization).

Training happens in two supervised stages: first on tool-free chain-of-thought examples (three epochs) to build RTL knowledge, then on the tool-integrated demonstrations (five epochs) to teach how to issue EDA requests and use what comes back. The toolset is Icarus Verilog, Verilator, and Yosys, plus a rule-based equivalence checker reused from CodeV-R1.

For reinforcement learning, they kept only tasks the SFT model failed at least once out of five attempts (about 8K tasks: roughly 7K generation/completion, 1K optimization), so the policy has room to improve. PAPO then rewards trajectories based on how they were checked: succeeding on the first task-appropriate check, succeeding after feedback-driven revision, or failing to check successfully. Rewards are gated on final verification, so an incorrect design gets zero regardless of its checking history. Task grouping keeps a 7:3 ratio of generation/completion to optimization trajectories in each update so the rarer optimization tasks are not drowned out.

Why This Matters

For research, the paper shows that the interaction pattern with tools — not just final answer correctness — is a usable training signal for RL, and that supervision ordering (reasoning first, tool use second) matters measurably at fixed token budgets. It also demonstrates that a smaller specialized model can beat a much larger tool-enabled teacher on a hardware benchmark.

Real-world applications:

• Automated RTL drafting and self-repair during chip front-end design, where a model compiles, simulates, and fixes its own Verilog before a human reviews it.
• Design optimization under constraints, such as cleaning lint violations or improving synthesis-based PPA proxies while preserving behavior — the paper's optimization task type.
• Legacy design completion, where masked or missing logic in an existing module is reconstructed under equivalence checking.
• Verification scaffolding, since the model authors its own simulation testbenches rather than relying on hand-written ones.

Industry relevance: the efficiency results (63.8% fewer tool calls, 28.8%–33.5% shorter responses) directly affect the cost of running an EDA-in-the-loop agent, where each tool invocation and long context has real compute and latency cost. The single-model design also removes the deployment overhead of coordinating separate generation and repair models or adapters.

Future Directions

• Repository-level RTL development: The authors explicitly note the work is limited to module-level tasks and has no protocol yet for cross-module dependencies, build configurations, and system-level verification.
• Improving trajectory data quality: CoT and synthesized TIR data may retain errors, and TIR quality is bounded by the teacher's capability and the coverage of automated checks; stronger validation of synthesized trajectories is named as future work.
• Broadening the automated check suite: The current acceptance criteria rely on simulation, lint, synthesis, and equivalence checking; extending coverage would widen the space of learnable checking patterns.
• Generalizing the pattern-aware reward: Whether the ISR/FRS/US reward structure transfers to other tool-using domains beyond EDA is left open.

Target Audience

Researchers and engineers working on LLM post-training, RL with tool use, or AI for EDA and hardware design. It is most useful to readers who already understand supervised fine-tuning and policy-gradient methods and want a concrete recipe for turning tool feedback into a training signal. Practitioners building RTL-generation assistants will find the benchmark numbers and the tool-efficiency results directly actionable; newcomers to hardware design will find the task taxonomy (generation, completion, optimization) a useful framing but will need background in simulation, linting, and synthesis to follow the acceptance criteria.

Authors’ abstract

Large language models offer a promising route to register-transfer-level (RTL) design, but reliable implementations require checking and revision based on electronic design automation (EDA) tools' feedback. Existing RTL-specialized models often lack learned interaction with such EDA tools or rely on separate generation and repair components. We introduce RTL-Interact, a multi-task post-training framework that trains a single RTL-specialized model to generate RTL, construct task-appropriate EDA checks, and revise implementations using tool feedback. The framework combines automatically constructed multi-task demonstrations with staged supervision from chain-of-thought to tool-integrated reasoning. We further introduce Pattern-Aware Policy Optimization (PAPO) for multi-task reinforcement learning. PAPO augments Group Relative Policy Optimization with correctness-gated rewards that favor concise successful checking while rewarding effective revision, and uses task grouping to preserve coverage across objectives. On CVDP, RTL-Interact-32B achieves 48.7% weighted Pass@1 and 68.9% Pass@5, surpassing its teacher model, DeepSeek-V3.2, while RTL-Interact-8B achieves 44.9% weighted Pass@1. Additionally, we show that PAPO can reduce average tool calls by 63.8% while improving Pass@1. Both models also outperform all RTL-specialized baselines on VerilogEval v2 and RTLLM v2, demonstrating the effectiveness of our method.