Back to the Future: Rethinking EDA Infrastructure for Agentic Systems in Chip Design Verification

AI summary

Overview

Research area: AI for Electronic Design Automation (EDA), specifically LLM-driven agents for chip design verification (waveform debugging, assertion checking, and root-cause analysis).

Technical level: Advanced. The paper assumes familiarity with RTL verification, Value Change Dump (VCD) waveforms, SystemVerilog Assertions (SVA), and text-to-SQL agents.

Scope: The paper proposes and evaluates BTTF (Back-to-the-Future), an end-to-end agentic framework that converts large simulation waveforms into a relational SQLite database so that multi-agent LLM systems can answer natural-language verification queries.

What This Paper Is About

LLM research for chip design is heavily skewed toward generating static RTL code, leaving post-simulation waveform debugging largely untouched because raw waveform dumps are far too large and unstructured for a model's context window. The authors name this barrier "Infrastructural Impedance" and argue that the real bottleneck is data infrastructure rather than model capability. Their goal is to make waveforms queryable by agents, turning an intractable context problem into a routine text-to-SQL problem.

Key Contributions

1. A waveform preprocessing pipeline that ingests proprietary binary .fsdb or ASCII .vcd dumps and normalizes them into a relational SQLite database (waves.db) with two tables: signal_metadata (static declarations such as RTL signal name and variable type, keyed by id) and signal_changes (timestamped (time, value) transitions with a foreign key back to metadata).
2. Coverage-aware pruning policies applied during on-the-fly database generation: clock pruning, inactive-signal pruning (a zero policy dropping statically silent zero-toggle wires, and an avg policy additionally dropping nets below the design-wide average toggle rate), and batched insertion.
3. A collaborative multi-agent orchestration engine with an Orchestration Agent that decomposes queries and routes them to an Analysis Agent (schema-aware SQL over the two tables), a Verification Agent (evaluating SVA rules ASSERT, ASSERT_STATIC, ASSERT_NEVER, ASSERT_KNOWN, COVER via regex assignment), and an Evaluation Agent that audits candidate queries against five criteria.
4. A benchmark and evaluation of 150 expert-annotated natural-language verification queries spanning debugging, analysis, and formal verification, plus a comparison against a monolithic single-agent baseline.

Main Findings

• Benchmark accuracy of 95.33%. Across the full 150-query taxonomy, BTTF reaches 95.33% overall execution accuracy, described as near-perfect on debugging primitives and consistently reliable on harder analysis and verification tasks.
• Clock pruning is highly effective. Removing clock transitions shrinks footprint from 1,420 MB to 480 MB (66.20% reduction), cuts query latency from 410.5 ms to 125.2 ms (a 3.28x reduction), and preserves 98.51% of functional toggle coverage.
• Inactive-signal pruning pushes reduction further. Zero-toggle pruning reaches a 395 MB footprint (72.18% reduction), 94.8 ms latency, and 100.00% coverage over active nets.
• Aggressive pruning has a hard boundary. Below-average pruning compresses to 185 MB (86.97% reduction) and 41.6 ms, but sacrifices 35.77% of coverage (64.23% coverage remaining), which the authors mark as the boundary of safe reduction.
• Batched insertion removes the ingestion bottleneck. Batched insertion via SQL EXECUTEMANY accelerates database generation by up to 2.89x over single-row insertion; the reported batch timings are 240.5 s (Batch 1), 100.2 s (Batch 10), 85.4 s (Batch 50), and 83.2 s (Batch 100).
• Task-level accuracy varies by category. Scope/Net Discovery 100.00%, Value Change Count 100.00%, Event Timing Check 100.00%, Value Inspection 95.00%, Toggle Coverage 91.67%, Power Hotspot 91.67%, Dynamic SVA Check 90.91%, RTL Attribution 87.50%, and Static SVA Check 84.62%.
• Multi-agent beats single-agent. A monolithic single agent scored only 3.35/5.00 on the Evaluation audit, suffering context clutter, schema hallucination, and degraded causal tracing on multi-hop verification tasks; BTTF scored 17.3% higher on reasoning quality, at the cost of an average 1.96x latency overhead.
• Assertion failures can be traced to specific nets. When a rule such as ASSERT_NEVER or ASSERT_KNOWN fails, the agent extracts the exact violation timestamp and backward-traces antecedent operands; the paper's example diagnostic is "Assertion failed at t=6929 ns due to net req_sel transitioning to unknown state x while clk was high."

Methodology in Plain English

The researchers start from a practical observation: an engineer debugging a failing simulation must manually scrub through huge temporal dumps in a waveform viewer, and an LLM agent cannot participate because the raw data is orders of magnitude larger than any context window and has no query interface.

Their fix is to change the data's shape rather than the model. An offline pipeline extracts the target hierarchies from the simulation dump and writes transitions into a SQLite database. Two design choices keep that database small: clocks are skipped at insertion time (since every row is already timestamped, temporal anchoring is preserved), and signals that never toggle, or toggle below the design-wide average, are dropped using bookkeeping maintained during ingestion — a vcd_id_to_db_id hash map and a per-signal signal_change_count counter. Writes are batched rather than row-by-row.

On top of the database, an online orchestration engine runs a team of specialized agents. The Orchestration Agent breaks a natural-language query into sub-goals and routes them. The Analysis Agent writes SQL joins across the two tables for scope discovery, event inspection, and toggle counting, using an iterative function-calling loop for multi-cycle statistics such as power hotspots. The Verification Agent ties anomalies back to version-controlled RTL by evaluating SVA rules. The Evaluation Agent audits answers for correctness, completeness, efficiency, schema adherence, and context understanding before any root cause is returned. Agents share a dynamic session context across turns, and a self-correction loop handles failed queries so each agent stays narrowly specialized without losing global coherence.

All agents are served through the Gemini API with Gemini-2.5-Pro as the default model. The evaluation draws on multiple SoC design hierarchies of varying complexity totaling 4.2 GB of VCD and 18,450 nets, with simulation windows ranging from 50 ns high-resolution glitch-debug windows to 800 ns full-transaction dumps.

Why This Matters

Impact on research. The paper reframes the agentic verification problem: the binding constraint is data infrastructure, not model capability. It positions relational database machinery (indexing, aggregation, low-latency filtering) as directly accessible to language agents once waveforms are made relational, and it treats data interfaces as a first-class research object in AI-driven chip design rather than an implementation detail. It also highlights that roughly 74.6% of LLM-for-hardware studies target static RTL generation, while post-simulation verification remains underexplored.

Real-world applications:

• Post-simulation debug: answering queries such as "What is the value of signal ctrl_status at time 7155316?" or "When is the first time the signal req_accept[7] is set to 42?" against billion-transistor designs.
• Assertion-based verification: running static and dynamic SVA checks and, on failure, automatically isolating the violation timestamp and the responsible sub-signal.
• Power and activity analysis: detecting power hotspots by iterative rolling-window queries (an example output reports a peak activity window of [7612 ns, 9944 ns] with 27 total signal toggles) and computing toggle coverage (an example reports 14.70% coverage, 21 active nets out of 143 total nets in scope).
• RTL attribution: mapping a waveform signal back to its source file and line (for example, design/.../moduleA.sv:L142 declaring logic [7:0] cnt).

Industry relevance. Verification consumes over 70% of the chip development cycle and is described as the primary bottleneck in modern chip design, while functional bugs that escape into full-chip regression or post-silicon testing carry exponential financial and schedule penalties. A framework that removes manual waveform scrubbing from the critical path addresses a cost center that grows with every generation of SoC complexity, and the authors argue the recipe generalizes beyond waveforms to regression logs, coverage databases, and power traces.

Future Directions

1. Streaming ingestion and standardized scope-path normalization to scale from extracted simulation windows to larger full-chip designs.
2. A waveform-to-RTL symbol index to strengthen the correlation between temporal anomalies and versioned source repositories.
3. Evaluating larger frontier reasoning models on multi-hop causal tracing over these relational stores.
4. Extending the pattern beyond waveforms — normalizing other opaque EDA artifacts such as regression logs, coverage databases, and power traces into queryable stores with specialized agents on top.

Target Audience

This paper is most useful to EDA and verification engineers seeking to automate post-simulation debug, AI/LLM researchers working on agentic systems and text-to-SQL over domain-specific data, and chip design tooling teams who own the interfaces between design data and automated analysis. Readers with a background in RTL verification or database-backed agent design will get the most from the implementation details; the closing argument about rebuilding EDA infrastructure to meet agents halfway speaks directly to infrastructure and methodology researchers deciding where to invest in AI-driven chip design.

Authors’ abstract

The unprecedented computational scale of modern artificial intelligence depends on complex, multi-billion-transistor Systems-on-Chip, yet the workflows that verify these chips remain stubbornly manual. Although Large Language Models (LLMs) have made rapid inroads into Electronic Design Automation (EDA), approximately 74.6% of existing studies target static Register-Transfer Level (RTL) code generation, leaving post-simulation verification and interactive waveform debugging largely untouched. We introduce Back-to-the-Future (BTTF), an end-to-end agentic framework that closes this infrastructural gap. BTTF distills massive, unstructured simulation dumps into a normalized relational SQLite database and couples it with a collaborative multi-agent orchestration engine that translates natural-language verification queries into schema-aware SQL while correlating signal anomalies with versioned RTL repositories. Across a 150-query benchmark, BTTF attains 95.33% execution accuracy, charting a practical path toward autonomous EDA verification.