Co-Layout: LLM-driven Co-optimization for Interior Layout

AI summary

Overview

Research area: Automated interior design — combining large language models with combinatorial optimization for computer vision / graphics and architectural layout generation.

Technical level: Advanced. The paper assumes familiarity with integer programming (big-M formulations, flow constraints, warm starts), LLM agent pipelines, and common generative-model evaluation metrics (CLIP, Q-Align).

Scope: The paper proposes a framework that reads a natural-language design brief, converts it into structured constraints via LLM agents, and solves for room layout and furniture placement jointly on a shared grid.

What This Paper Is About

Interior design tools typically break the job into two separate stages: first decide where the rooms go, then decide where the furniture goes. That split causes problems, because furniture determines what a room is for and the room's size and shape determine what furniture fits. This paper asks an LLM to extract the design constraints from a user's written request, then solves room layout and furniture placement together as a single optimization problem so the two decisions are consistent with each other.

Key Contributions

1. An automated interior design framework that simultaneously optimizes room layout and furniture placement while satisfying constraints generated by LLMs.
2. A grid-based spatial representation that formalizes LLM-derived constraints — corridor connectivity, room accessibility, spatial exclusivity, adjacency, furniture containment — as an integer programming problem.
3. A coarse-to-fine optimization strategy that solves a simplified low-resolution problem first and uses that solution to guide the full-resolution solve, reducing computation time.
4. A multi-agent LLM preprocessor that converts a text prompt into a structured scene graph (rooms, furniture, dimensions, spatial constraints), plus heuristic post-processing for walls, doors, and windows.

Main Findings

• Physical plausibility: On the five examples in Figure 4, the method records an object overlap rate (OOR) of 0.00 and an out-of-boundary rate (OOB) of 0.00, matching AnyHome's OOR of 0.00 while improving on its OOB of 0.04, and far better than Holodeck's OOR of 0.82 and OOB of 2.33.
• Image quality and aesthetics: Q-Align scores are 4.17 (image quality assessment) and 3.35 (image aesthetic assessment), versus 4.10/3.32 for AnyHome and 4.03/3.32 for Holodeck.
• Text-image alignment: CLIP similarity is 26.50, compared with 25.75 for AnyHome and 25.15 for Holodeck.
• User study ratings: Across 64 valid responses from 71 participants, the method scores 3.77 (semantic alignment), 3.23 (layout rationality), and 3.41 (path clearance) on a 5-point scale, versus 3.43/3.12/3.06 for Holodeck and 3.07/2.59/2.80 for AnyHome.
• User study ranking: Mean Reciprocal Rank is 0.80 for the proposed method, 0.59 for Holodeck, and 0.45 for AnyHome.
• Circulation failures in baselines: Holodeck's layouts sometimes require passing through a bedroom to reach the main living area, and AnyHome can generate entirely unreachable rooms or irregular, narrow room shapes.
• Coarse-to-fine efficiency: In an ablation with five runs per setting using a fixed prompt and varying grid resolution, the strategy significantly reduces computation time as resolution increases while maintaining comparable solution quality. The paper gives the illustrative example of down-sampling a 12×10 grid to a 6×5 grid, where each coarse cell represents a 2×2 block, reducing problem scale by 75%.
• Demonstrated input range: Generated examples include two apartments, a small coffee shop, a 100 m² clinic, and a 150 m² clinic, covering both residential and non-residential spaces.

Methodology in Plain English

The floor is treated as a 2D grid, where each cell can be labeled as corridor, as belonging to one specific room, or as covered by one specific piece of furniture. A "Modulor"-inspired idea from classical architecture supplies the basic unit of this grid. Each furniture item is a rectangle with dimensions the LLM chooses, and its orientation is restricted to one of four axis-aligned directions selected by two binary variables.

A layer of specialized LLM agents imitates an architect's process: one analyzes the basic floor envelope and family structure, one reasons about climate and orientation, one places outdoor space and the entrance, one enumerates rooms with target areas and relationships, and one lists furniture with dimensions and arrangement preferences. Their output is a structured specification rather than coordinates, because LLMs cannot reliably produce precise coordinates and often produce conflicts.

That specification becomes an integer program. Constraints include non-overlap and full coverage of the floor, corridor connectivity enforced through a flow formulation from the entrance cell, room accessibility (at least one furniture-free cell adjacent to corridor or open-room space), adjacency between specified room pairs, room bounding boxes, furniture containment inside its room, and constraints tying each furniture item's occupied cell count to its area. Doors and windows are not optimization variables; they are placed by post-processing.

Quality is driven by a weighted sum of penalty terms: room rectangularity, room perimeter (compactness), deviation from target areas, aspect ratio control, relative positioning between furniture pairs, and balance between the area-weighted furniture centroid and the room's geometric centroid.

Because the problem grows exponentially with grid resolution, the method first solves a down-sampled version using only room layout variables, maps that solution back to the fine grid as a warm start, and adds a reference penalty that discourages the fine solution from deviating from the coarse room assignments. The warm start is not guaranteed feasible, which is why the penalty mechanism is included.

Experiments use Python with the GUROBI solver, version 12, on a laptop with an 8-core Ryzen processor and 16 GB of RAM. Baselines Holodeck and AnyHome are used only to generate layout information, converted to a unified format, and rendered with identical visualization scripts. Final layouts are exported as scene white boxes in Blender, and assets are retrieved from the 3D-FUTURE and Imaginarium libraries via semantic embedding and size matching.

Why This Matters

Impact on research: The paper argues that room layout and furniture placement are inherently interdependent and that existing pipelines treat them as separate stages, making synergistic consideration difficult. It also claims that earlier floorplan representations — rasterized and vectorized alike — struggle to express high-level constraints like corridor connectivity and room accessibility, whereas a grid-based representation makes them explicit. This reframes interior design as a constraint-satisfaction problem where LLMs supply semantics and a solver supplies geometry.

Real-world applications:

• Residential floorplan drafting, where a written brief about family structure and lifestyle becomes a first-pass apartment layout.
• Commercial space planning such as the coffee shop example, where room functions and equipment must fit a fixed footprint.
• Professional facilities such as the 100 m² and 150 m² clinics shown, where functional zoning and clearance matter.
• Design-iteration tooling that lets a designer explore many prompt variants before committing to a scheme.

Industry relevance: The framework targets productivity gains for interior designers by automating the tedious constraint-balancing portion of the job, and it demonstrates an approach where an LLM handles high-level reasoning while a solver handles the precise coordinates that LLMs cannot reliably produce. This division of labor is directly relevant to any product that wants to generate buildable geometry rather than plausible-looking images.

Future Directions

• Extending the furniture scope beyond floor-based objects to wall-mounted items and tabletop objects, which the current framework does not cover.
• Handling conflicting constraints produced by the LLM, which the authors say can degrade solution quality; they suggest iterative refinement or human-in-the-loop feedback.
• Extending the framework to multi-story buildings by incorporating inter-floor constraints.
• The supplementary material also promises further ablations on key constraints and on the co-optimization approach itself, suggesting that isolating the contribution of each constraint family remains an open investigation.

Target Audience

Researchers and practitioners in computer vision, computer graphics, and computational design who work on floorplan generation and indoor scene synthesis; optimization specialists interested in integer-programming formulations of spatial layout; and applied engineers building LLM-agent systems that must output precise, physically valid geometry rather than text alone. The paper is dense with mathematical notation, so readers without a background in integer programming will find the formulation sections demanding, while the introduction, overview, and experimental sections are accessible to a general technical audience.

Authors’ abstract

We present a novel framework for automated interior design that combines large language models (LLMs) with grid-based integer programming to jointly optimize room layout and furniture placement. Given a textual prompt, the LLM-driven agent workflow extracts structured design constraints related to room configurations and furniture arrangements. These constraints are encoded into a unified grid-based representation inspired by ``Modulor". Our formulation accounts for key design requirements, including corridor connectivity, room accessibility, spatial exclusivity, and user-specified preferences. To improve computational efficiency, we adopt a coarse-to-fine optimization strategy that begins with a low-resolution grid to solve a simplified problem and guides the solution at the full resolution. Experimental results across diverse scenarios demonstrate that our joint optimization approach significantly outperforms existing two-stage design pipelines in solution quality, and achieves notable computational efficiency through the coarse-to-fine strategy.