Design Anything, Design at Scale

AI summary

Overview

Research area: Agentic AI for multimodal design generation — specifically the "harness" (the software layer around a large language model that manages tools, context, and multi-step execution) and benchmark design for long-horizon creative work.

Technical level: Intermediate. Readers should be comfortable with the idea of LLM agents, tool calls, and evaluation benchmarks, but the paper's core mechanism (an explicit task state plus a controller that checks deliverables) is explained in plain terms.

Scope: The paper proposes Design Anything, a graph-based harness that keeps a design brief's constraints and decisions outside the model and binds each requested deliverable to the workflow steps that produce it, together with DesignAnything-Bench, a 228-task benchmark across 19 design domains used to test whether agents stay aligned with design intent as projects grow from 6 to 24 artifacts.

What This Paper Is About

Real design projects start from a brief that fixes goals, constraints, and a list of required deliverables, and later artifacts are supposed to build on decisions made earlier — a selected concept, a supplied reference, an agreed palette. Current image and video generators can make individual artifacts well, but agents running long multi-step projects tend to stop early, lose track of constraints, and drift away from earlier decisions as the number of deliverables multiplies.

The paper's goal is to show that much of this drift can be fixed outside the model: by recording the design intent in an explicit, checkable state, wiring each requested deliverable to the specific workflow steps that produce it, and using a deterministic controller — not the model's own claim of completion — to decide when a run is actually finished.

Key Contributions

1. Design Anything harness. A harness whose central object is an intent–execution graph: a task contract holding deliverables, constraints, and recorded design decisions, a typed directed acyclic workflow graph that the agent edits only through validated tool calls, and execution records. A binding maps each deliverable to the workflow nodes that produce it, so a selected concept or reference enters later generation as an explicit input rather than being re-described from scratch.

2. A delivery loop with a deterministic controller. After every turn, the controller derives which deliverables are still outstanding from the bindings, node execution states, and checks on output kind and item count. A run ends successfully only when no deliverables remain, no jobs are running, and no questions or decisions are pending — so the model saying it is done is explicitly insufficient.

3. DesignAnything-Bench. A benchmark of 228 tasks spanning 19 design domains (12 tasks per domain), including 38 project families posed at 6, 14, and 24 requested artifacts with shared background and inputs (114 matched-scale tasks), plus 57 reference-material tasks and 57 breadth tasks, representing 152 distinct projects.

4. Evaluation with the LLM held fixed. Head-to-head comparisons against OpenDesign and JarvisHub on DeepSeek V4.1 Flash High and against Codex CLI on GPT-5.6 Sol High, plus a delivery-loop ablation, matched-scale scaling analysis, per-domain analysis, and a blind study with professional designers.

Main Findings

• Constraint satisfaction, overall: With the LLM held fixed, Design Anything satisfies 85.0% of the brief's factual constraints with DeepSeek V4.1 Flash High and 92.5% with GPT-5.6 Sol High, exceeding the strongest same-model baselines by 15.4% and 14.4% respectively.

• Advantage grows at scale: On the 38 matched project families at 24 artifacts, Design Anything reaches 85.7% (DeepSeek) and 91.9% (GPT), leading OpenDesign by 25.1% and Codex by 23.0%. These are the same gains cited as 25.1% and 23.0% on 24-artifact tasks.

• Delivery is not the same as compliance: On 24-artifact tasks, Codex delivers over 97% of requested artifacts but satisfies 68.9% of constraints, versus 91.9% for Design Anything with the same LLM. Because a constraint counts only if it holds in every eligible artifact, this gap reflects fulfillment across the set rather than file counts.

• Broad domain coverage: Under each model, Design Anything leads or ties in 18 of 19 domains on constraint satisfaction (12 tasks per domain). JarvisHub leads in product design on DeepSeek, and Codex in exhibition design on GPT.

• Other automatic measures: Design Anything leads its LLM group on reference fidelity (75.6% DeepSeek, 79.3% GPT) and identity consistency (84.5%, 90.2%). It does not lead on set consistency: OpenDesign scores 88.3 vs 87.9 on DeepSeek, and Codex scores 89.8 vs 87.7 on GPT. Codex also has higher point estimates for delivery completion (98.1 vs 97.3), technical quality, and text rendering. JarvisHub leaves 12.4% of artifacts undelivered.

• Ablation on the delivery loop: On 19 paired 24-artifact tasks with DeepSeek V4.1 Flash High, stopping when the model stops calling tools gives 73.9% delivery and 57.5% constraint satisfaction, with 6 runs below half delivery. The full harness (Complete) reaches 96.1% delivery and 90.4% constraint satisfaction, eliminating all six low-delivery runs.

• Cheaper than nagging: A "Generic prompt" configuration that answers every stop by telling the model to keep working achieves numerically similar delivery (96.9% vs 96.1%) but issues 36.1 generation requests per run versus 24.1 for the full harness, at roughly one-third the mean LLM cost: $0.43 versus $1.30 on 17 paired tasks with complete telemetry.

• Designer preference: Twelve designers in five disciplines made 1,019 blind pairwise judgments over 185 tasks. Designers chose Design Anything over OpenDesign in 82% of judgments, over Codex in 70%, and over JarvisHub in 51% (JarvisHub 37%, 12% ties). Bradley–Terry estimates place the GPT configuration first at 74.0% and the DeepSeek configuration second at 60.7%, both with 95% intervals above 50%.

• Qualitative failure modes remain: In a 14-artifact animation brief where two otters swap matching cases, JarvisHub turns the otters into humans in its color script, Codex reuses one image for two action phases, and Design Anything on DeepSeek leaves the cases unreturned in its final keyframe — showing that credit for delivering files does not guarantee correct content.

Methodology in Plain English

The design separates intent from generation. A structured task contract stores the brief's goals, deliverables, constraints, success criteria, open questions, and decisions marked as coming from the user or the agent. It deliberately holds no completion counter, so editing it changes what is wanted rather than recording progress.

A typed directed acyclic graph holds the workflow. Each node computes an output from its own configuration plus the outputs of its parent nodes, and edges connect compatible output and input ports. When an artifact should build on a selected concept or a supplied reference, that dependency becomes an explicit edge, so the later generation consumes the actual prior output instead of a fresh text description. A binding connects each contract deliverable to the set of nodes meant to produce it.

The model works in turns and can only act through four families of tools — PLAN (record or revise intent), BUILD (create or edit nodes, edges, and bindings), EXECUTE (run nodes), and AUDIT (read state). Every operation passes a deterministic validity check covering tool arguments, node configurations, references, port types, and bindings; rejected operations leave the state untouched and return an actionable error.

When a turn ends, the controller decides what is unfinished: a deliverable counts as delivered only if it has a binding, all its bound executable nodes have finished, and its outputs belong to that deliverable and pass checks on output kind and item count. Output-kind checks rely on declared node capabilities, so an image that merely looks like a document cannot satisfy a document deliverable. If anything remains, the controller starts another turn that names the outstanding deliverables and their blocking reasons. The run stops successfully only when no running jobs, no pending questions or decisions, and no outstanding deliverables remain; repeated turns that show no observable progress are stopped by progress limits.

Evaluation is done from exported artifacts, independent of the system's internal representation, using an Agent-as-a-Judge protocol. Measures cover delivery completion, constraint satisfaction, reference fidelity, identity consistency, set consistency, and design quality (an image checklist mean multiplied by delivery completion), with judge calibration against human annotations reported in the appendix.

Why This Matters

Research impact. The paper reframes long-horizon agent quality as a property of the harness rather than of the model alone. By holding the LLM fixed and varying only the surrounding system, it isolates how much constraint-following can be recovered through explicit state, validated tool operations, and deterministic completion checks. The matched-scale benchmark design — the same 38 project families at 6, 14, and 24 artifacts — makes degradation with project size measurable, which single-scale benchmarks cannot show.

Real-world applications:

• Studio and agency production, where one client brief spawns concept images, renderings, spec documents, decks, and video that must stay mutually consistent.
• Brand and packaging work, where names, dimensions, palettes, and typography must hold across every artifact in a set.
• Exhibition, architectural, and interior projects, where drawings, spatial views, and documents must derive from an agreed concept.
• In-house marketing and e-commerce pipelines, where product imagery, advertising sets, and campaign assets must match supplied reference objects.

Industry relevance. The economic argument is as important as the accuracy one: the full harness matched a naive "keep working" prompt on delivery while using roughly one-third the LLM cost ($0.43 vs $1.30 per run). For teams shipping agentic design tools, that suggests structured delivery tracking is a cost lever, not just a quality lever. The public release of the benchmark, evaluator, dataset, and project page also gives vendors a common yardstick for a claim that is otherwise easy to assert and hard to verify.

Future Directions

• Extending beyond the recorded domains and scales. The benchmark covers 19 domains and three scales (6, 14, 24 artifacts); whether the advantage holds for substantially longer projects, for artifact types not represented, or for non-English briefs is not established here.

• Closing the remaining content-level failures. The qualitative comparison shows errors that delivery tracking cannot catch: substituted subjects, reused images across distinct items, and unresolved narrative details. The paper notes that constraint satisfaction and consistency are evaluated from exported artifacts, leaving open how the harness should verify semantic content within an artifact rather than only its kind and count.

• Narrowing the gaps where baselines still lead. OpenDesign and Codex score higher on set consistency, and Codex has higher point estimates for delivery completion, technical quality, and text rendering. Whether those measures can be raised without sacrificing the constraint-satisfaction lead is an open engineering question.

• Understanding where the baselines win locally. JarvisHub leads in product design on DeepSeek and Codex in exhibition design on GPT. Explaining these domain-specific reversals would clarify which parts of design intent most need external state.

Target Audience

Researchers and engineers building LLM agents for multi-step creative or production work, especially those working on harness design, context management, and completion verification. Also useful for benchmark designers interested in measuring degradation as task scale grows, for teams evaluating design-automation products who want a same-model comparison methodology, and for design practitioners curious about what agentic systems currently get right (file delivery, constraint adherence) and wrong (content fidelity, cross-artifact consistency). Readers need no specialist mathematics; the paper is equation-light and the mechanisms are described operationally.

Authors’ abstract

As multimodal resources are transformed into media outputs, agentic AI is expected to stay aligned with design intent over long horizons. Designers follow briefs specifying goals, constraints, and deliverables, with later work building on earlier decisions. Recent work reuses design experience across tasks, yet alignment among artifacts within one project remains challenging. Agents therefore need to preserve and apply this intent across domains and as projects grow. We introduce Design Anything, a graph-based harness that preserves design intent outside the model and binds deliverables to the workflow steps that produce them through an intent-execution graph. A task contract retains constraints and decisions, selected designs become explicit inputs to later generations, and a deterministic controller uses bindings and execution records to check output types and counts, identify outstanding deliverables, and guide subsequent turns. We also introduce DesignAnything-Bench, with 228 tasks across 19 design domains, including 38 project families at scales of 6, 14, and 24 artifacts. With the LLM held fixed, Design Anything satisfies 85.0% and 92.5% of the brief's factual constraints with DeepSeek V4.1 Flash High and GPT-5.6 Sol High, respectively, exceeding the strongest same-model baselines by 15.4% and 14.4%. These gains reach 25.1% and 23.0% on 24-artifact tasks. Under each model, Design Anything leads or ties in constraint satisfaction in 18 of 19 domains. Professional designers also prefer its complete deliverable sets over those of same-model baselines in a blind evaluation.