CRAB: Cuboid-based Representation for As-built BIM from Point Clouds

AI summary

Overview

Research area: Computer vision and computer graphics, specifically 3D scene reconstruction and Scan-to-BIM (converting laser scans of buildings into editable Building Information Models). Published at SIGGRAPH Asia 2026 Conference Papers.

Technical level: Advanced — the method combines an autoregressive point-cloud-to-sequence transformer with a Flow Matching refinement network, and assumes familiarity with point cloud processing and BIM/IFC concepts.

Scope: The paper proposes CRAB, a framework that reconstructs indoor buildings from point clouds as sets of parametric, semantically labeled, relation-aware cuboids representing walls, doors, windows, beams, and columns, and releases a companion dataset (CrabSet) for training such models.

What This Paper Is About

Scanning a building produces a noisy, incomplete point cloud — raw surface measurements with no notion of "wall thickness" or "this door belongs to that wall." A BIM model, by contrast, is a set of editable, semantically typed, volumetric components with explicit relationships. CRAB's goal is to bridge that gap directly, predicting architectural components as parameterized cuboids with BIM-compatible semantics and structure, rather than stopping at the intermediate abstractions (2D floor plans or 3D planes) that most prior methods produce.

Key Contributions

1. A BIM-oriented cuboid representation. The paper recasts indoor Scan-to-BIM as semantic cuboid assembly. Each component is parameterized by a bottom centerline (two endpoints), a thickness, a height, and a semantic label — a "centerline-based" formulation that explicitly separates a wall's longitudinal extent from its thickness, unlike generic oriented boxes parameterized by center, size, and yaw.

2. CrabSet, a cuboid-level supervision dataset. Because existing indoor datasets lack component-wise volumetric cuboid annotations, the authors extend the public Structured3D dataset with cuboid annotations for architectural components and add procedurally synthesized scenes with simulated point-cloud acquisition. CrabSet totals 6,500 synthetic scenes across four subsets: S3D-derived, Boundary-varied, Scale-varied, and Corridor-centered.

3. A proposal-and-assembly framework. ProposalNet (a Sonata point-cloud encoder plus a Qwen2.5-0.5B autoregressive generator) predicts coarse semantic cuboids; AssemblyNet, a relation-aware Flow Matching model, jointly refines cuboid geometry and infers inter-component relations such as wall adjacency and door/window-wall affiliation, while supporting human-guided local edits.

4. A complete path to editable BIM. Refined cuboids and predicted relations are converted into IFC-compatible components, with wall junctions resolved by trimming or extending walls along their longitudinal axis, rooms assembled from closed cycles of at least four wall segments, and openings attached to their host walls. The same cuboid-relation graph can produce Corner-Joint, Cross-Joint, or Wall-Integrated BIM styles.

Main Findings

• Accurate wall thickness, the paper's headline result: CRAB reaches a mean wall-thickness error of 2.80 cm on interior walls, with 79.50% of matched walls within 3 cm (T-Acc@3). The strongest non-cuboid baseline, SpatialLM* (retrained with ground-truth opening heights), reaches 41.62% T-Acc@3.

• Better component matching than floor-plan and plane alternatives: On the Structured3D test set, CRAB achieves 93.78% Match-F1 on exterior walls, 92.57% on interior walls, 91.14% on doors, and 88.97% on windows. RoomFormer reaches 84.48% / 85.78% / 77.05% / 64.82% on the same categories; CAGE reaches 91.14% / 89.74% but does not report door or window results.

• Much better opening heights: Door height error is 2.86 cm and window height error is 9.23 cm for CRAB, versus 77.36 cm and 122.86 cm respectively for SpatialLM. Window Match-F1 improves from 77.90% to 88.97% over SpatialLM.

• Dataset diversity helps: The variant trained only on S3D-derived cuboids (Ours*) already beats baselines but scores lower than the full model (e.g., 88.07% vs. 91.14% door Match-F1, 3.35 cm vs. 3.01 cm interior-wall centerline offset), indicating the procedural scenes add value.

• No sacrifice in layout or plane quality: When CRAB's output is converted into 2D room polygons and 3D planes for the baselines' native metrics, it reaches 98.7 room F1, 93.1 corner F1, and 91.1 angle F1, and improves window F1 from 87.5% to 91.4% at IoU₂D@0.5 over SpatialLM.

• Cuboids are more stable refinement targets than planes: In a controlled wall-perturbation test, plane-based refinement produced endpoint errors of 30.29 cm, 13.07 cm, and 33.21 cm under length-plus-axial, perpendicular, and combined perturbations; cuboid-based refinement reduced these to 3.60 cm, 5.18 cm, and 5.66 cm.

• Full-scene refinement helps cuboids but hurts planes: Using SpatialLM's planes as AssemblyNet input degraded results (interior-wall T-Acc@3 of 20.45%), while cuboid-based AssemblyNet improved interior-wall T-Acc@3 from 79.50% to 81.27% and raised exterior-wall Match-F1 to 94.42%.

• Generalizes to real scans: On the real-world BIMNet benchmark, CRAB scores 72.5%, 68.8%, and 48.5% Match-F1 for walls, doors, and windows, versus 68.6%, 49.6%, and 37.4% for SpatialLM — the largest gains on openings.

• Beams and columns are supported but unevenly: Evaluated separately on procedurally generated scenes, CRAB reaches 97.15% Match-F1 for columns but only 57.83% for beams.

• Centerline parameterization matters for walls: Replacing it with a generic oriented box drops interior-wall T-Acc@3 from 79.50% to 71.74% and exterior-wall Match-F1 from 93.78% to 92.59%, while oriented boxes are slightly better on the compact door category (92.20% vs. 91.14%).

Methodology in Plain English

The approach is split into two stages, with the first deliberately kept simple.

Stage one — propose components. A point cloud is voxel-downsampled at 2.5 cm and capped at 800,000 points. A pretrained Sonata encoder turns it into features, a small MLP adapter maps those features into a language-model token space, and Qwen2.5-0.5B generates cuboids one after another as a token sequence, up to 200 cuboids, stopping at an end-of-sequence token. This stage only worries about individual components; it does not reason about how they connect.

Stage two — assemble and correct. Each proposed cuboid is turned into a set of tokens: its geometric and semantic parameters, one special "relation token," and an edit flag marking whether it is fixed or editable. Three attention mechanisms process these tokens — one looking within a cuboid, one passing information between cuboids (so a wall can sense its neighbors' positions, sizes, and categories), and one cross-attending to the point cloud so refinement stays grounded in the actual scan. A velocity head then predicts incremental geometric corrections over K Flow Matching steps, accumulating updates from the coarse proposals; the relation tokens go to a separate relation head that predicts pairwise links (door-to-wall, wall-to-wall adjacency) using an order-invariant feature built from sums, absolute differences, and element-wise products of the two tokens. The edit flag is what makes human-in-the-loop use possible: a user can add, delete, move, or anchor cuboids, and only the editable ones are changed.

Stage three — convert to BIM. Predicted adjacencies are used to trim or extend walls at junctions without altering their direction, thickness, or centerline offset. Closed loops of at least four wall segments become rooms. Openings are attached to host walls. The result is instantiated as IFC elements.

Training setup: 3,000 Structured3D training scenes plus 3,000 procedural scenes, 50 epochs, Adam with an initial learning rate of 1×10⁻⁴, cosine decay with 1% warm-up, on 8 NVIDIA RTX 4090 GPUs and dual Intel Xeon Gold 6430 CPUs.

Why This Matters

Research impact. The paper argues that representation choice — not just model capacity — is the bottleneck in learning-based Scan-to-BIM. By showing that cuboids beat floor plans and planes on BIM-critical attributes like wall thickness and opening height without losing layout quality, it reframes the task: predict the editable components directly, rather than predicting an abstraction and inferring components afterward. The controlled perturbation and full-scene refinement experiments provide concrete evidence that cuboids are more stable optimization targets than planes. The released dataset also addresses a real supervision gap, since the authors' comparison table shows most existing indoor datasets provide only 2D floor plans, 3D planes, or semantic labels, not volumetric components.

Potential real-world applications:

• Renovation and retrofitting: Capturing an existing building as editable, thickness-accurate wall and opening geometry is a prerequisite for planning modifications.
• Construction progress monitoring: Comparing as-built scans against design models requires components with identity and parameters, not just surfaces.
• Facility management and digital twins: Semantic components with adjacency relations can carry attributes such as load-bearing status or materials through the building's lifecycle.
• Automated floor plan and documentation generation: The paper notes that the same cuboid-relation graph can be converted into 2D floor plans or 3D planar representations, and that rooms are assembled from wall-adjacency cycles.

Industry relevance. Output is IFC-compatible, the open BIM exchange standard, and supports multiple junction conventions (corner-joint, cross-joint, wall-integrated). The human-in-the-loop editing path addresses a practical reality: fully automatic reconstruction from incomplete scans is not always achievable, so a system where a user seeds or anchors components and the model regularizes them may be more deployable than a fully automatic one.

Future Directions

• Recovering entirely missing components. The authors state that CRAB can correct continuous geometric errors, local misalignments, and redundant components automatically, but completely missing elements may still require coarse user initialization. Closing this gap — inferring what is absent from partial scans — is the clearest open problem.

• Irregular and curved geometry. CRAB handles Manhattan layouts and moderate non-Manhattan variations, but highly irregular or curved structures can be oversimplified by cuboids. Extending the representation to curved or free-form elements without losing editability is unresolved.

• Richer component coverage and benchmarks. Beams, columns, stairs, and service shafts appear sparsely in real-world test scenes, which limits evaluation — the beam result (57.83% Match-F1) is notably weaker than columns (97.15%). The authors call for richer component-level supervision and broader BIM benchmarks, which would also let these categories be evaluated against reproducible baselines.

• Expanding the relation graph. The current relation set covers door/window-wall affiliation and wall-wall adjacency. Attributes the introduction mentions as motivations — load-bearing status and materials — are not modeled, and the paper frames a component-first representation as the route to carrying such attributes.

Target Audience

Researchers and graduate students in 3D vision, computer graphics, and remote sensing working on indoor scene reconstruction, structured modeling, or primitive/cuboid-based shape representations. It also suits practitioners in architecture, engineering, and construction (AEC) evaluating automated Scan-to-BIM, and engineers building digital-twin or IFC pipelines who want to understand what a learned component-level representation can and cannot yet deliver. Readers need comfort with point cloud architectures, transformers, and diffusion/flow-matching ideas; the BIM vocabulary (IFC, wall thickness, junction types) is introduced in enough context to follow.

Authors’ abstract

As-built BIM reconstruction from point clouds demands semantic, volumetric, parametric, and editable building components, beyond 2D layouts or planar abstractions. Yet existing scan-to-BIM pipelines often rely on handcrafted rules, heuristic optimization, or manual correction, while learning-based methods fall short of recovering wall thickness, opening geometry, and component-level BIM structure. We introduce CRAB, a BIM-ready cuboid reconstruction framework that encodes walls, doors, windows, beams, and columns as compact architectural cuboids with IFC-compatible semantics and parameters. CRAB bridges point clouds and editable BIM by combining ProposalNet, which predicts coarse component cuboids, with AssemblyNet, a relation-aware Flow Matching model that refines geometry, predicts structural relationships, and enables automatic structural correction and human-guided editing. To address limited cuboid-level supervision, we extend Structured3D with component-wise cuboid annotations and construct cuboid-based procedural scenes. Experiments show CRAB surpasses floor plan and plane-based alternatives in component completeness, geometric accuracy, and generalization. For interior wall thickness estimation, CRAB achieves a mean error of 2.80 cm, with 79.50% of matched walls within 3 cm. The project page is available at https://vcc.tech/research/2026/CRAB.