Architect-Ant: Editable Automatic Furnishing of Architectural Floor Plans

AI summary

Overview

• Research area: Artificial intelligence applied to architectural/interior design — automatic furniture layout generation from 2D floor plans, combining dataset curation, supervised fine-tuning, and reinforcement learning.
• Technical level: Advanced. The paper assumes familiarity with large language/vision-language models, LoRA fine-tuning, GRPO reinforcement learning, and layout geometry metrics.
• Scope in one sentence: The paper introduces AntPlan (a dataset of 505 real professional floor plans with dense furniture annotations) and Architect-Ant (a framework that fine-tunes and then reinforces a pretrained model to emit editable, constraint-aware furniture layouts as a coordinate-based DSL, which can be converted to 3D scenes).

What This Paper Is About

Furnishing a floor plan automatically is hard because furniture placement must satisfy many interacting geometric and functional constraints (a bed must fit, but it also affects storage access and circulation), and because real professional floor-plan data with dense furniture labels is scarce. The authors ask whether professional furnishing knowledge can be learned from real plans by a pretrained model, so that constraint-aware layouts can be generated directly rather than through costly iterative agentic reasoning. Their goal is a system that produces layouts which are geometrically valid, functionally complete, and still editable object-by-object.

Key Contributions

1. AntPlan dataset: 505 real professional architectural floor plans with structural, room, and dense furniture annotations spanning 92 object classes and ten residential room categories — described as substantially expanding furniture-class coverage over existing 2D floor-plan datasets.
2. Architect-Ant framework: A system that adapts pretrained model knowledge through supervised fine-tuning and rule-guided reinforcement learning to generate editable DSL layouts for downstream 3D scene construction.
3. Layout Rule Score (LRS): A room-aware score that converts geometric and functional constraints into layout-level reinforcement-learning rewards, enabling constraint-aware generation without prescribing intermediate reasoning steps.
4. Direct generation without an inference-time critique loop: At inference, the model generates candidate layouts and ranks them with LRS, rather than running a model-based critique-and-revision loop.

Main Findings

• Best functional completeness and LRS on the benchmark: On 109 released SceneSmith room shells, Architect-Ant reaches FUN+ 94% and LRS 8.80, the highest among compared methods. SceneSmith reaches FUN+ 91 and LRS 8.44; Holodeck 78 and 7.84; LayoutVLM 86 and 6.29; LayoutGPT 84 and 2.10; ATISS 84 and 1.50; InstructScene 63 and −2.82; DiffuScene 71 and −3.55.
• Low geometric violations: Architect-Ant reports COL+ 0.6% and OOB 0.6%. Holodeck reports 0.0% for both, SceneSmith 3.4% and 1.3%, LayoutGPT 8.6% and 8.5%.
• High accessibility but not the top: Architect-Ant's accessible-pair rate is 97.2%, within 2.3 points of the best method (Holodeck, 99.5%).
• Not achieved by sparse layouts: Architect-Ant produces the highest mean number of furniture objects per room among the evaluated furnishing categories (10.8 objects/room; 5.45 objects per 10 m²), with coverage of 29.1%.
• Much faster than agentic methods: Architect-Ant is reported at 95.4 s per scene, versus 272.0 s for LayoutVLM, 432 s for Holodeck, and a 1,555 s object-count-normalized estimate for SceneSmith (not a measured rerun on filtered objects). ATISS is 20.2 s, InstructScene 21.6 s, DiffuScene 74.3 s, LayoutGPT 67.7 s.
• Both visual judges prefer Architect-Ant: Claude Sonnet 5 prefers Architect-Ant in 72 of 109 rooms (72 wins / 1 tie / 36 SceneSmith wins) and Kimi K3 in 74 of 109 (74 / 1 / 34). The strongest preference is on kitchen/dining shells; SceneSmith is preferred in four of the six bathrooms.
• SFT and GRPO both help: On the 243 held-out AntPlan rooms, mean LRS rises from the base direct-DSL model to SFT, and again with reasoning-enabled GRPO in all four room types. Mean LRS after SFT + GRPO reasoning: kitchen 3.48 ± 1.82, bathroom 7.00 ± 0.74, living room 6.22 ± 0.92, bedroom 6.02 ± 0.75 (base: −11.71 ± 3.62, −0.53 ± 1.35, 2.29 ± 1.50, −1.51 ± 1.58).
• Best-of-eight improves further: Best-of-8 LRS after SFT + GRPO reasoning is 5.80 ± 1.70 (kitchen), 9.00 ± 0.43 (bathroom), 8.95 ± 0.53 (living room), and 8.79 ± 0.46 (bedroom).
• Qualitative differences beyond the numbers: The authors report that Holodeck may obstruct doors or windows, SceneSmith can produce sparse arrangements or implausible orientations, and the learned baselines show visible object overlaps in several cases.
• LRS is not an aesthetics measure: The paper states LRS captures only the geometric and functional properties encoded by its rules; it does not measure aesthetics or guarantee full 3D usability. A score of 10 means no encoded rule was violated, not that a layout is ideal.

Methodology in Plain English

The authors first build a dataset from published professional architectural drawings. Walls, doors, windows, and railings are extracted with an RT-DETR-X detector trained on CubiCasa5K; room categories are labeled manually; a furniture detector bootstrapped from a hand-labeled subset proposes object boxes that are then manually reviewed and corrected. The authors describe these reviewed furniture annotations as pseudo-labels rather than exhaustive ground truth. Each room is cropped with a 0.5 m context margin, and pixel geometry is converted to room-local meters using a scale estimated from the median annotated door-opening width, assuming 0.8 m — an approximate calibration rather than a reading of the drawing's dimension text.

Each layout is written as a compact sequence of objects, each with a class, a top-left footprint corner, and width and height in meters — a domain-specific language that keeps object-level editability and lets containment, overlap, and clearance be checked directly.

The model starts from Gemma-4-31B-it with a LoRA adapter trained per room type (rank 64, scaling 128, dropout 0.05) while the vision encoder stays frozen. Supervised fine-tuning for 5 epochs teaches the furnishing domain and the DSL, using only final layouts as targets — no reasoning traces. Then GRPO reinforcement learning optimizes the final layout against a deterministic room-aware score, LRS, which starts at 10 and subtracts penalties for violated rules. The evaluator implements 33 rule types covering shared structural checks (containment, wall overlap, door and window obstruction, disallowed overlap, inventory mismatch, accessibility) and room-specific functional relationships; typical validation rooms require roughly 50–200 predicate evaluations. Reward comes only from the parsed final layout, so intermediate reasoning gets no direct supervision and the model develops its own strategy.

Training uses 16 completions per room and eight rooms per update (128 on-policy rollouts) followed by one policy-gradient step, for 15 updates total, with learning rate 2×10⁻⁵ and KL coefficient 0.001 against the frozen pretrained base; advantages are clipped to [−2, 2]. For a new room, the system retrieves similar same-type AntPlan rooms to form furniture requests, generates candidates, and selects the highest raw LRS; the chosen layout is then converted to a furnished 3D scene by retrieving compatible assets and inferring orientation from nearby walls and furniture.

Why This Matters

• Research impact: The paper argues that professional layout examples, explicit spatial constraints, and pretrained model knowledge can be combined during training instead of at inference, avoiding iterative agentic refinement loops. It also contributes a dataset whose furniture-class coverage (92 classes) is far finer than existing 2D floor-plan datasets such as CubiCasa5K's 10 coarse furniture classes, and a reusable rule-based scoring function (LRS) for evaluating layouts on geometric and functional criteria, not just visual quality.
• Real-world applications (as named in the paper):
  • Real-estate visualization and listing imagery.
  • Interior design, including exploring and communicating alternative furniture arrangements.
  • 3D scene construction for architectural visualization.
  • Robotics simulation and game development, which need furnished scenes.
• Industry relevance: The pipeline turns an empty structured plan into an editable, object-level layout that can be rendered as a 3D scene, and does it in roughly 95.4 s per room on two NVIDIA H100 80GB GPUs, versus the much larger runtimes reported for iterative agentic systems. Editability matters commercially because designers can adjust individual objects rather than regenerating a whole scene.

Future Directions

• Extending the framework to additional object and scene properties, richer geometry, and more detailed spatial relationships while preserving structured, editable generation (stated explicitly in the conclusion).
• Going beyond LRS's scope: because LRS measures only encoded geometric and functional rules, not aesthetics or full 3D usability, developing evaluation or rewards for those properties is an open problem.
• Improving the weak spots identified: the visual judges preferred SceneSmith in four of the six bathrooms, and Architect-Ant's accessible-pair rate (97.2%) is below the best baseline (99.5%), suggesting room for improvement in specific room types and accessibility.
• Broadening the data: AntPlan covers four modeled room types in the experiments (bedroom, bathroom, kitchen, living room), and the benchmark excludes some room types for the learned 3D-FRONT baselines, leaving other residential categories and non-residential plans unexplored.
• Note: the provided paper text is truncated mid-appendix, so additional reported results may exist that are not summarized here.

Target Audience

Researchers and practitioners in computer vision, graphics, and AI for design who work on indoor scene synthesis, floor-plan understanding, or LLM post-training with verifiable rewards. It is also relevant to interior designers, real-estate visualization teams, and simulation/game developers who need automatic furnishing, and to dataset builders interested in annotation workflows that combine detectors with human review. The paper is written at an advanced technical level: readers will get the most from it with background in transformer-based models, LoRA fine-tuning, and GRPO-style reinforcement learning.

Authors’ abstract

Furnished floor plans support real-estate visualization, interior design, and architectural workflows, yet automatic furnishing remains challenged by limited real-world data and the need to satisfy interacting geometric and functional constraints. We ask whether professional furnishing knowledge can be learned from real floor plans using a pretrained model, enabling direct constraint-aware layout generation without relying on costly iterative agentic inference. We introduce AntPlan, a curated dataset of 505 real professional architectural floor plans with dense furniture annotations spanning 92 object classes and ten residential room categories, and Architect-Ant, a framework for generating furniture layouts. Architect-Ant represents layouts with an editable coordinate-based DSL and first learns professional furnishing patterns through supervised fine-tuning. It is then optimized with GRPO using a Layout Rule Score (LRS) that aggregates geometric and functional constraints derived from professional plans, providing outcome-level supervision without prescribed reasoning traces. Experiments against diverse state-of-the-art baselines show that Architect-Ant combines low geometric violation rates with high functional completeness, while qualitative results more closely reflect real-world residential furnishing patterns. The resulting layouts remain object-level editable and can be converted into 3D scenes.