CANDO: Cooperative Agentic Network for Layout Design Optimization

AI summary

Overview

Research area: Multi-agent systems applied to generative layout design — specifically, constraint-aware spatial layout synthesis for real-world facilities.

Technical level: Advanced. The paper combines constrained rigid-body optimization over 2D polygonal domains, Voronoi-based motion planning, multi-agent LLM orchestration, and a purpose-built benchmark with a multi-part scoring protocol.

Scope: This paper introduces ALPS-Bench (a benchmark of 1,000 professionally annotated real-world facility layouts with a constraint-grounded scoring protocol) and CANDO (a training-free cooperative multi-agent framework that refines layouts through a verification-grounded loop), then evaluates CANDO on ALPS-Bench and on PubLayNet, RICO, and PKU-PosterLayout.

What This Paper Is About

Existing layout-generation benchmarks mostly target simple placements over rectangular canvases and score outputs with distributional metrics such as FID and IoU, which reward resemblance to the dataset prior rather than actual design quality. The authors argue this penalizes valid but atypical designs and does not transfer to real facility layout, where irregular site boundaries, heterogeneous orientations, access-point-aware placement, and vehicle routing feasibility all matter at once. Their goal is to build a benchmark capturing those real constraints and a cooperative agentic method that can satisfy them.

Key Contributions

1. ALPS-Bench (Asset Layout Planning and Synthesis Benchmark): 1,000 real-asset, professionally annotated facility layouts, each paired with a structured design manual of spatial constraints, plus an instance-specific scoring protocol that scores compliance on a held-out test set rather than prior matching. Annotations were produced by professional annotators working from original CAD design drawings.

2. CANDO (Cooperative Agentic Network for Design Optimization): described as, to the authors' knowledge, one of the first cooperative agentic AI frameworks for constraint-aware layout synthesis. It combines structured agent cooperation, tool-grounded verification, episodic feedback memory, block-wise best-of-N selection, and motion planning into a single training-free framework.

3. State-of-the-art results on public benchmarks: a simplified CANDO achieves state-of-the-art mIoU and alignment on PubLayNet and RICO, and leading overlap and strict-underlay performance on PKU-PosterLayout, supporting the claim that cooperative agentic design is a broadly effective recipe for layout generation.

4. Evidence that dataset-specific training is insufficient: two baselines trained or fine-tuned specifically on ALPS-Bench (LayoutDM adapted and trained from scratch, and Qwen3-VL-8B-Instruct fine-tuned with LoRA-SFT) are substantially outperformed by CANDO on the ALPS-Bench test set.

Main Findings

• Large AGG gains on ALPS-Bench: CANDO (CoDe/Qwen3.6) reaches AGG 0.62 ± 0.19, CS 0.83 ± 0.25, and RE 0.41 ± 0.16, versus 0.40 ± 0.27 / 0.47 ± 0.37 / 0.32 ± 0.22 for the Single-Agent (GPT-5.2) baseline. CANDO (CoDe/GPT-5.2) reaches AGG 0.60 ± 0.25, CS 0.78 ± 0.35, RE 0.42 ± 0.17; CANDO (BoN/GPT-5.2) reaches AGG 0.59 ± 0.25, CS 0.77 ± 0.36, RE 0.41 ± 0.17.

• Trained baselines fail on hard constraints: Qwen-LoRA-ALPS scores AGG 0.07 ± 0.12, CS 0.01 ± 0.08, RE 0.13 ± 0.22, and LayoutDM scores AGG 0.13 ± 0.13, CS 0.01 ± 0.03, RE 0.24 ± 0.25. LayoutDM frequently violates geometric hard constraints because its bounding-box representation only approximates the true polygonal site geometry, while the single-pass Qwen-LoRA-ALPS captures high-level compositional patterns but struggles with containment, non-overlap, and routing feasibility.

• LLM-based baselines also lag: LayoutPrompter (GPT-5.2) reaches AGG 0.17 ± 0.11 and LayoutCoT (GPT-5.2) reaches AGG 0.19 ± 0.20 on ALPS-Bench.

• Fewer refinement steps than the single-agent baseline: CANDO variants use T = 20 refinement steps versus T = 200 for the Single-Agent baseline, and require 5.09 or 8.55 or 8.81 average iterations to comply (CANDO CoDe/Qwen3.6, BoN/GPT-5.2, CoDe/GPT-5.2 respectively) versus 80.6 for Single-Agent. Compliance rates are 74.0%, 60.6%, and 63.6% for the three CANDO variants versus 22.4% for Single-Agent.

• Competitive time complexity despite more operation per step: assuming a single LLM inference call costs τ, and applying a ×4 factor for sequential agent calls, CANDO costs 20.36τ to 35.24τ versus 80.6τ for the Single-Agent baseline. LayoutPrompter and Qwen-LoRA-ALPS cost τ, and LayoutCoT costs 4τ, but at far lower AGG.

• Statistical significance: paired, instance-level two-sided Wilcoxon tests with Holm correction and paired-bootstrap 95% confidence intervals over the 100 test instances show both CANDO variants significantly outperform all considered baselines on AGG, CS, and RE (all p_Holm ≤ 0.0059), while the Qwen3.6 and GPT-5.2 CANDO variants are statistically comparable across all three metrics (p_Holm = 1.000).

• Perfect alignment and high mIoU on PubLayNet and RICO: On Gen-T, CANDO reaches mIoU 0.866 with FID 2.101, alignment 0.000, overlap 0.005 on RICO, and mIoU 0.893 with FID 3.455, alignment 0.000, overlap 0.000 on PubLayNet. On Gen-TS, CANDO reaches mIoU 0.879 / FID 1.371 / alignment 0.000 / overlap 0.052 on RICO and mIoU 0.881 / FID 2.023 / alignment 0.000 / overlap 0.000 on PubLayNet. The highest prior mIoU figures in the table are LayoutCoT's 0.719 (RICO Gen-T), 0.772 (PubLayNet Gen-T), 0.795 (RICO Gen-TS), and 0.847 (PubLayNet Gen-TS).

• FID is not consistently competitive: despite leading on mIoU, CANDO ranks second or last on FID depending on the setting. The authors state this inconsistency is symptomatic of a broader limitation, though that discussion is cut off in the provided content.

• Overlap improvements over LayoutCoT: across all five RICO tasks, CANDO improves overlap over LayoutCoT in four cases, with Gen-TS as the exception.

• Failure modes of non-agentic and prior-matching approaches: the paper attributes the poor hard-constraint performance of prior methods to reliance on bounding-box approximations and to single-pass generation without iterative, tool-grounded verification.

Methodology in Plain English

The problem setting. A layout is a 2D polygonal site (the "premise") with access points (entrance, exit) on its edges, plus a fixed set of elements — each with a semantic class and a fixed canonical shape — that must be placed. Only each element's centroid position and rotation angle are optimized; shapes, classes, and counts are given. Hard constraints require containment inside the premise, no overlaps, and routing feasibility for a set of predefined vehicle scenarios. Soft constraints reward compositionality (arrangement patterns and distance thresholds), orientation (permitted orientations and intra-group alignment), and proximity (stacking, clearance, distance bounds). The objective is a weighted sum of soft-constraint compliance, maximized subject to validity.

The benchmark. ALPS-Bench contains 1,000 annotated layouts from real operational sites, split into 900 development instances (released with full annotations, including ground-truth configurations) and 100 held-out test instances (released with input requirements only, ground truth withheld for evaluation on a hosted server). Elements come from 14 semantic classes across four functional groups, and not all 14 appear in every layout.

The scoring protocol. Hard Constraint Metrics check element existence and sizing, containment, non-overlap, and routing feasibility — with missing or incorrectly sized elements zeroing the score. Routing feasibility is assessed with a Voronoi-based motion planner that inflates obstacle borders by a scenario-dependent safety margin, samples seed points, and uses Voronoi medial axes to find maximum-clearance paths. Soft Constraint Metrics check proximity, compositionality, and orientation, with −0.1 penalties per violation for proximity and compositionality and continuously scaling penalties for angular deviation. Ground Truth Metrics compare generated layouts against withheld ground truth on path length, maneuverability (average number of turns), and path clearance, each normalized to [0, 1] with 0.5 meaning equivalence to ground truth. These are combined into CS, RE, and an aggregate AGG score.

How CANDO works. It runs in two phases. First, an in-context learning initializer embeds the new input requirements into a feature vector covering per-class element counts and area coverage, access-point counts and positions, element size statistics, and layout descriptors (area, perimeter, compactness), retrieves the most similar development layouts with a weighted kNN search, and generates several candidate seed layouts in parallel, ranking them with a geometric proxy score that is 1 exactly when containment and pairwise non-overlap hold. Second, four specialized agents cooperate in a parallel-branching refinement loop adapted from CoDe's blockwise sampling scheme: a Verifier scores candidates with deterministic tools and turns diagnostics into feedback; a Historian maintains a sliding window of feedback changes; a Planner drafts a remediation plan, optionally decomposed across specialist workers with a supervisor reconciling them using a multi-modal context (diagnostic feedback, history, geometric coordinates, and a rendered plot); and a Generator produces new candidates using a strategy-diversified ensemble. A block-wise best-of-N reduction selects the best branch every B steps. Crucially, low-level geometry is offloaded to deterministic tools rather than left to the language model, because LLMs struggle with spatial reasoning and coordinate arithmetic. The Verifier shares ground-truth-free design-manual checks with the benchmark evaluator, but final scores are recomputed independently and the optimization never sees a held-out target layout.

Experiments. CANDO is evaluated on ALPS-Bench plus PubLayNet, RICO, and PKU-PosterLayout. For PubLayNet and RICO the authors evaluate type-conditioned generation, type/size-conditioned generation, and (in the appendix) relational generation, refinement, and completion, using adjusted mIoU, alignment, overlap, and FID. Most experiments use Qwen3.6 as the agent backbone (a Mixture-of-Experts model with 35B total parameters and 3B active parameters), with GPT-5.2 also tested via API. Quantitative evaluations used NVIDIA H100 GPUs with 80 GB of VRAM.

Why This Matters

Impact on research. The paper argues that distributional metrics such as FID, IoU, SSIM, and graph similarity implicitly penalize design innovation by scoring valid-but-atypical configurations down for departing from the empirical distribution, collapsing the task into prior matching. ALPS-Bench replaces prior matching with instance-specific constraint compliance and routing efficiency, and CANDO demonstrates that a training-free cooperative agentic loop can beat models trained specifically on the target benchmark.

Real-world applications:

• Facility layout planning for energy and industrial sites, where the premise boundary is irregular and the design must reflect as-built CAD geometry.
• Placement of service infrastructure such as variant service units sized for different vehicle classes, electric vehicle charging areas, underground service facilities, and underground access points.
• Parking layout design across car, bicycle, priority, and heavy-goods vehicle bay variants.
• Access-aware placement that must preserve collision-free routes for multiple vehicle scenarios entering and exiting through designated access points.

Industry relevance. The paper is authored by researchers affiliated with Shell Information Technology International, Shell China Limited, and Delft University of Technology

Authors’ abstract

Layout generation for real-world facilities is a challenging problem, requiring reasoning over irregular site boundaries, heterogeneous orientations, access-aware placements, and motion-planning feasibility. Yet, most existing layout benchmarks in the generative AI space target simpler placements over rectangular domains and rely on distributional metrics such as FID and IoU that reward conformity to dataset priors, thus discounting design innovation. Motivated by these gaps, we introduce ALPS-Bench, a benchmark of $1,000$ professionally annotated real-world facility layouts paired with an instance-specific scoring protocol grounded in a structured design manual. As a strong baseline for ALPS-Bench, we propose CANDO, a training-free multi-agent framework in which specialized agents iteratively refine layouts through a verification-grounded loop, concentrating reasoning on strategic spatial decisions. We demonstrate that CANDO surpasses state-of-the-art trained and LLM-based baselines on the widely adopted PubLayNet, RICO, and PKU-PosterLayout benchmarks, establishing cooperative agentic design as a broadly effective recipe for constraint-aware layout synthesis.