CircuitLM: A Multi-Agent LLM-Aided Design Framework for Generating Circuit Schematics from Natural Language Prompts

AI summary

Overview

Research area: Artificial intelligence applied to electronic design automation (EDA) — specifically, multi-agent large language model (LLM) pipelines that convert natural-language hardware requests into structured, electrically verifiable circuit schematics.

Technical level: Intermediate (assumes familiarity with LLMs, retrieval-augmented generation, and basic electronics/EDA concepts such as pinouts, ERC, and netlists).

Scope (one sentence): The paper presents CircuitLM, a five-stage multi-agent framework that grounds LLM circuit generation in a curated component knowledge base to produce a machine-readable "CircuitJSON" schematic, and evaluates it across five frontier LLMs on 100 embedded-systems prompts using a deterministic electrical rule checker plus an LLM-as-a-judge meta-evaluator.

What This Paper Is About

LLMs can answer "build a gesture-control circuit with an ESP32-C3 Mini and five MPU6050 sensors," but the output is typically unstructured prose that hallucinates components, omits essential supporting parts (pull-ups, decoupling capacitors, current-limiting resistors, I²C multiplexers), ignores power constraints, and cannot be loaded into standard EDA tools like Fritzing or Proteus Design Suite. The goal of CircuitLM is to replace that unreliable prose with a structured, physically grounded, machine-readable schematic format, and to provide an evaluation methodology that can actually measure whether a generated circuit is electrically safe and functional.

Key Contributions

1. CircuitLM pipeline: A five-stage multi-agent pipeline (component identification, canonical pinout retrieval, chain-of-thought design reasoning, JSON schematic synthesis, and interactive force-directed visualization) that converts natural-language prompts into visually interpretable schematics.
2. CircuitJSON: An open, tool-agnostic schematic interchange format for structured hardware representation, specifying component placement (coordinates, rotation), an attrs map for parametric values, and canonicalized pin-to-pin connections.
3. A deterministic ERC evaluation framework and benchmark: A 100-prompt dataset plus a custom graph-based Electrical Rule Checking engine that parses CircuitJSON and categorizes faults by severity (Critical/Fatal, Major, Minor, Warning), validated by blind expert review.
4. A retrieval-augmented architecture that eliminates component hallucination: By decoupling prompt context from total database size (N), the framework scales to industrial component libraries with O(k) prompt complexity, where k is the number of components required for a given design.

Main Findings

• High deterministic ERC pass rates: Pass@1 rates under the ERC engine were 88% (Gemini), 87% (Qwen), 86% (Deepseek), 84% (Llama), and 83% (GPT). Gemini and Qwen led.
• Fatal errors were nearly eliminated: Average fatal errors were μ = 0.0 for nearly all backends, which the authors attribute to the chain-of-thought planning phase forcing models to identify VCC/GND rails. GPT was the only model reported with non-zero average fatal errors (μ = 0.0 ± 0.2), described as still extremely low frequency.
• Major errors remained the primary failure mode: These involved logic-level mismatches (for example, 5V vs. 3.3V interfacing) and missing current-limiting resistors in multi-component systems.
• Deepseek showed a distinct minor-error/warning profile: μ = 0.3 ± 0.9 for warnings, often flagged for missing decoupling capacitors on active components.
• A large "evaluation gap" between ERC and LLM-as-a-judge: Gemini passed ERC at 88% but only 53% under the LLM judge; Llama fell from 84% to 21%. The judge caught UART RX/TX confusion, SPI protocol miswiring, missing bus pull-ups, use of non-PWM-capable pins for PWM, pin multiplexing conflicts, and undervoltage or brownout risks that the rule-based checker missed.
• Strong inter-rater reliability: Human experts versus the ERC engine achieved Fleiss' kappa κ = 0.82 (95% CI [0.74, 0.93]), with near-perfect agreement on safety-critical tiers (Fatal κ = 0.94, Major κ = 0.88) and lower agreement on subjective tiers (Minor κ = 0.73, Warning κ = 0.68). Blind audits of the LLM judge's outputs yielded κ = 0.78 (95% CI [0.65, 0.88]).
• Difficulty stratification exposes hard-case weakness: ERC scores stayed relatively flat across Easy, Medium, and Hard prompts (for example, Gemini 90%, 80%, 92%), while LLM-judge scores collapsed on Hard prompts (Gemini 75%, 60%, 40%; Llama 45%, 36.67%, 2%).
• Ablation shows large context-efficiency gains: The zero-shot full-context baseline consumed roughly 15,000 tokens per request versus roughly 3,000 tokens for CircuitLM, at a latency cost for CircuitLM (roughly 8 seconds vs. roughly 3 seconds due to multiple API calls). Llama-3.3 consistently failed in the zero-shot setting due to immediate context overflows and was excluded from that ablation.
• Baseline comparison is mixed: Moving from zero-shot to CircuitLM raised ERC Pass@1 for Gemini (+4%), Qwen (+8%), and Deepseek (+9%), but lowered it for GPT-5 mini (-2%). The authors hypothesize that for distilled, highly optimized models, overly prescriptive planning instructions may interfere with internal heuristic efficiencies.
• Deepseek had the widest overall profile: Deepseek showed a notable overall kappa of 0.86 under ERC and 0.40 under LLM judging, with the only reported score above 0.5 under the LLM judge for Deepseek. (Compare: overall LLM-judge Pass@1 was 53% Gemini, 40% Deepseek, 38% Qwen, 23% GPT, 21% Llama.)

Methodology in Plain English

The authors built a pipeline of specialized agents, each handling one narrow job, rather than asking a single model to do everything at once:

1. Identification Agent reads the user's prompt and extracts the hardware list using named entity recognition plus intent analysis, outputting generic component categories (for example, "Microcontroller," "Motor Driver," "DC motor") instead of exact part numbers, to keep the search space small.
2. Retrieval Agent takes that list and runs embedding-based similarity search against a local vector knowledge base of canonical components, augmented with fuzzy lookup against a dictionary of equivalent parts. It returns verified canonical names and exact pin definitions. If a requested part is absent, an Out-of-Distribution flag halts execution and requests human review — an explicit safety guardrail against hallucinated wiring.
3. Electronics Expert Agent performs design reasoning, producing a chain-of-thought document that encodes functional goals, power requirements, safety constraints, and pin-level wiring logic, using least-to-most prompting to break complex requirements into hierarchical sub-tasks.
4. Circuit Generation Agent turns the chain-of-thought plus canonical pin mappings into a strict CircuitJSON object, specifying placement (coordinates and rotation) and canonical pin-to-pin connections, with an attrs field for parametric values such as resistance, capacitance, inductance, and voltage limits.
5. Schematic Visualizer renders CircuitJSON into interactive SVG in the browser, using a force-directed algorithm to place connected components without overlap and Manhattan-style wire routing that favors short paths and minimal bends. Users can drag components and reshape wires in real time.

The knowledge base is a ChromaDB vector store holding 70+ components, each with canonical name, mandatory pin labels, physical dimensions, aliases, description, category, typical usage, and technical specifications. Embeddings come from Qwen3-Embedding-0.6B with cosine similarity and an empirically determined threshold. A separate firmware code-generation agent (for ESP32 and Arduino targets) is included as a demonstration but is not part of the core workflow.

For evaluation, the authors discarded SPICE-style simulation as computationally infeasible and largely irrelevant for microcontroller-centric digital targets, and instead wrote a custom deterministic ERC engine that parses CircuitJSON into a topological graph using networkx. It performs programmatic pathfinding, models passive components as weighted traversal paths to distinguish direct shorts from resistive bridges, enforces logic-level matching (V_out > V_in_max), detects floating inputs and net contention, and validates constraints against a hardware knowledge base covering maximum operating voltages, current limits, pin multiplexing, and inductive load requirements. Faults are tiered as Fatal (safety-critical), Major (functional), Minor (performance), or Warning (best practices). The primary metric, Pass@1, requires zero Fatal and zero Major errors; OOD halts count as automatic failures.

Because ERC only verifies electrical connection rules and cannot confirm that a circuit actually works, a second QA Agent using Claude Sonnet 4.5 (excluded from the generator pool to avoid self-preference bias) acts as an LLM-as-a-judge on protocol correctness, logical routing, and firmware plausibility, assuming board-level reality where modules may include onboard regulators.

Five LLMs were benchmarked across the text-generation stages (Stages I, III, IV): gpt-5-mini, gemini-2.5-flash, deepseek-v3.1, qwen3-235b-a22b-2507, and llama-3.3-70b-instruct. All were accessed through a unified API with temperature fixed at 0, the same base model used across all agents within a run, and each model tested three times. Prompts came from a 100-prompt dataset stratified into Easy, Medium, and Adversarial tiers. Three electrical engineering experts, who completed a calibration phase using a predefined error taxonomy, conducted blind review of a stratified random sample of 25% of generated circuits from each of the five models, balanced across difficulty tiers.

Why This Matters

Impact on research: The paper argues that existing benchmarks such as PINS100 and MICRO25 lack holistic electrical validity checks, structured machine-readable outputs, and automated visualization, and that prior LLM hardware-generation work (VerilogEval, ChipChat, LLM4Netlist, Schemato, EESchematic, MenTeR) targets digital RTL, analog circuits, or RF/analog design rather than embedded systems. CircuitLM positions structured intermediate representation plus layered verification — a deterministic rule engine and a semantic LLM judge — as the way to evaluate generated hardware, and it explicitly frames CircuitJSON as a candidate standardized bridge between natural language and hardware description for future EDA datasets.

Real-world applications:

• Rapid prototyping of embedded systems, robotics, and IoT hardware from a spoken or written project idea, reducing the time from idea to a testable design.
• Educational electronics: lowering the barrier for novices who lack component knowledge or schematic drafting skills, with a lightweight visual-first workflow suited to early prototyping.
• Acting as a design assistant alongside simulation platforms such as Tinkercad or Wokwi, or feeding into Fritzing and Proteus workflows.
• Safety screening of LLM-generated wiring, where the ERC engine and the LLM judge together catch critical shorts, logic-level mismatches, and protocol-level mistakes before a board is built.

Industry relevance: The scalability argument is central: injecting an entire component library into a prompt consumed roughly 15,000 tokens versus roughly 3,000 for CircuitLM's targeted retrieval, and full-context injection becomes cost-prohibitive and breaches context windows as libraries grow toward industrial proportions. A retrieval design whose prompt complexity scales with the number of components in the design (k) rather than the size of the library (N) is what makes industrial-scale component catalogs plausible.

Future Directions

• Simulator-in-the-loop verification: The authors note the absence of automated Wokwi or Proteus co-simulation as a notable limitation of the evaluation framework.
• Transpilation to standard EDA formats: A direct transpilation layer from CircuitLM's connection arrays into Python-based SKiDL definitions, enabling generation of KiCad-compatible netlists. The paper notes that converting CircuitJSON into other representations, such as a netlist, currently requires robust scripting and additional engineering effort.
• Heterogeneous multi-LLM evaluation: A system using agreement-based consensus and iterative feedback loops to validate and correct connectivity logic before synthesis, along with ensemble architectures for scalability across complex industrial designs.
• Extension beyond digital/embedded electronics: The evaluation focused primarily on digital and embedded electronics; analog circuits such as op-amp-based designs were not experimentally assessed.
• Reducing multi-agent overhead: Computational overhead from iterative multi-agent queries and variable latency depending on model and API traffic remain open issues, as does the reliance on a single evaluator for LLM-based validation.

Target Audience

Researchers working at the intersection of LLMs and electronic design automation; engineers building agentic or retrieval-augmented pipelines for structured hardware or domain-constrained outputs; embedded systems and IoT prototypers interested in natural-language-to-schematic tooling; and educators or EDA tool developers evaluating machine-readable interchange formats and automated electrical rule-checking benchmarks. Readers need a working understanding of LLM agent design and basic circuit concepts (pins, pull-ups, logic levels, protocols like I²C, SPI, and UART) to follow the evaluation sections.

Authors’ abstract

Generating accurate circuit schematics from high-level natural language descriptions remains a persistent challenge in electronic design automation (EDA), as large language models (LLMs) frequently hallucinate components, violate strict physical constraints, and produce non-machine-readable outputs. To address this, we present CircuitLM, a multi-agent pipeline that translates user prompts into structured, visually interpretable $\texttt{CircuitJSON}$ schematics. The framework mitigates hallucination and ensures physical viability by grounding generation in a curated, embedding-powered component knowledge base through five sequential stages: (i) component identification, (ii) canonical pinout retrieval, (iii) chain-of-thought reasoning, (iv) JSON schematic synthesis, and (v) interactive force-directed visualization. We evaluate the system on a dataset of 100 unique circuit-design prompts using five state-of-the-art LLMs. To systematically assess performance, we deploy a rigorous dual-layered evaluation methodology: a deterministic Electrical Rule Checking (ERC) engine categorizes topological faults by strict severity (Critical, Major, Minor, Warning), while an LLM-as-a-judge meta-evaluator identifies complex, context-aware design flaws that bypass standard rule-based checkers. Ultimately, this work demonstrates how targeted retrieval combined with deterministic and semantic verification can bridge natural language to structurally viable, schematic-ready hardware and safe circuit prototyping. Our code and data are publicly available at https://github.com/Khandakar227/CircuitLM.