TopoField: Technical Report for ECCV 2026 SceneFun3D Challenge

AI summary

Overview

Research area: 3D computer vision, specifically instance-level segmentation of functional interaction elements (knobs, buttons, handles, sockets) in indoor point clouds.

Technical level: Intermediate. The paper assumes familiarity with point-cloud instance segmentation and test-time augmentation, but its core ideas — averaging many noisy predictions and grouping points by 3D connectivity — are intuitive.

Scope: A technical report describing TopoField, a deterministic, parameter-free post-processing decoder that converts the point-wise probability output of an existing segmentation model (VOLT) into the ranked instance masks the SceneFun3D benchmark requires, placing second on the ECCV 2026 SceneFun3D Challenge.

What This Paper Is About

The SceneFun3D benchmark asks a system to find nine categories of small, interactable parts inside complete indoor laser scans and return each one as a category label, a point mask, and a confidence score. Existing point-wise predictors output a continuous per-point probability for each class, but the benchmark scores a discrete, confidence-ranked list of instances, so a large gap exists between what the model produces and what is evaluated. TopoField's goal is to close that field-to-instance gap through two explicit deterministic stages — stabilizing the probability field with test-time augmentation and then decoding and ranking instances from the geometry of that field — without training any new parameters.

Key Contributions

1. Formulating and quantifying the field-to-instance gap. The authors frame the mismatch between continuous point-wise semantics and the benchmark's ranked instance-set output, and measure it with complementary statistics: whether any candidate in the pool is valid versus whether the top-ranked candidates are valid.

2. An index-aligned test-time semantic consensus stage. Thirteen forward passes (twelve from a 4-rotation × 3-scale grid plus a double-axis flip) are merged by retaining the processed-point index of every fragment prediction, accumulating softmax probabilities in a common N × 10 point-by-class array, normalizing by coverage count, and fusing the independent caches with normalized weights.

3. A geometry-aware instance decoding stage. Each class chooses among argmax connectivity, foreground-threshold connectivity, or a seeded graph basin. The basin branch separates high-confidence cores from a broader support region, builds a symmetric radius-pruned 32-nearest-neighbor graph, and uses multi-source graph-geodesic (Dijkstra) assignment to split support regions containing several seeds, followed by class-wise scoring and soft non-maximum suppression.

4. A parameter-free decoder with measured gains. The decoder adds no learnable parameters to VOLT and reaches 28.286% AP, 41.383% AP₅₀, and 47.558% AP₂₅ on the official test set, earning second place in the challenge.

Main Findings

• The candidate pool is far better than the top-ranked output. On 561 ground-truth instances across 30 validation scenes, an IoU-0.5-valid candidate exists for 67.2% of instances (R_any@0.5), but top-1 coverage is only 20.0% (R₁@0.5) — a 47.2-point gap. R₃@0.5 is 38.3 and R₅@0.5 is 46.3.
• Point-wise semantics are only moderately reliable. Foreground-point precision is 68.4% and recall is 60.1%, confirming that per-point probabilities are noisy for these small, partly occluded parts.
• Support/core selection is the largest single gain. On the 20-scene development split, adding dual-threshold support and core selection raises AP by 1.68 points (35.21 → 36.89).
• Graph-geodesic partitioning adds further separation. It contributes a further 0.71 development-split points (36.89 → 37.60), separating adjacent candidates that share material, color, and local geometry.
• Class-conditioned routing adds a third gain. It contributes 0.85 development-split points (37.60 → 38.45) and lifts the 10-scene holdout split from 41.86 to 42.31.
• Cumulative validation improvement. The full pipeline raises AP across all 30 validation scenes from 36.58 (baseline) to 38.14 (+ dual-threshold), 38.92 (+ graph geodesic), and 39.67 (+ class-conditioned).
• Official test results depend on checkpoint and post-processing. Epoch-80 robust: 27.919 AP / 41.373 AP₅₀ / 45.878 AP₂₅; epoch-80 aggressive: 27.640 / 41.103 / 46.364; epoch-100 robust: 28.270 / 40.847 / 46.761; epoch-100 aggressive: 28.286 / 41.383 / 47.558. The best configuration is epoch-100 with aggressive post-processing.
• The two stages are complementary. Stage 1 stabilizes the probability field across augmented views; Stage 2 converts that field into masks and orders them, so candidate construction and candidate ranking both matter.

Methodology in Plain English

The authors leave the underlying segmentation network (VOLT) frozen and build everything around its output.

First, they run the network thirteen times on transformed versions of the same scan — rotating it four ways around the vertical axis at three slightly different scales (twelve combinations), plus one pass with a double-axis flip. Each transformed pass is voxelized into fragments, and each fragment carries an index back to the original point it came from. Because several fragments can map to the same original point, the authors add up the predicted class probabilities per original point and divide by how many contributions it received. The thirteen resulting caches are then averaged with normalized weights and renormalized so each point's class scores sum to one. This produces one smooth, stable "consensus" probability field in the original point order.

Second, for each of the nine target categories, they collect the points whose most likely class is that category, and split them into a broad support set (probability at least 0.15) and a smaller, more trustworthy core set (probability at least 0.40), requiring at least five support points and at least three core points. They build a sparse graph over the support points only: connect each point to its 32 nearest support neighbors, keep edges in both directions, and drop any edge longer than a radius limit. Distances along graph edges are the point-to-point Euclidean lengths, floored at 10⁻¹². Connected components are found first; if a component contains more than one core cluster, a multi-source shortest-path search grows outward from all the seeds simultaneously and gives each support point to whichever seed reaches it along the shortest accumulated path. This keeps labels from jumping across empty space between separate objects.

Finally, each resulting mask is scored by the average consensus probability of its class over its points (quantile, hybrid, and a class-wise size-scaling factor are also supported, with the scaling exponent set to zero by default). Exact duplicate masks keep only the highest-scoring copy, and overlapping same-class masks are suppressed with soft NMS that exponentially decays a candidate's score by its IoU with a better one.

Why This Matters

The work shows that a meaningful part of the accuracy lost on fine-grained 3D part segmentation comes not from weak recognition but from the unglamorous step of turning probabilities into instances. That reframes where effort should go: proposal construction and ranking, not only backbone capacity. Its diagnosis — that 67.2% of instances have a valid candidate somewhere in the pool while only 20.0% are found at rank one — is a reusable measurement for anyone building decoders.

Real-world applications:

• Robotics and assistive manipulation: a robot told to "press the button" or "turn the knob" needs the exact extent of the small interactable part, not just the appliance it sits on.
• Augmented-reality instruction and maintenance: overlaying guidance on the correct handle or switch in a scanned room requires instance-level part masks in the room's own coordinate frame.
• Digital twins and smart-building annotation: automatically labeling functional elements in laser scans of interiors for facility management and simulation.
• Home safety and accessibility auditing: locating sockets, switches, and handles at scale in scanned environments.

Industry relevance is direct: the method adds no learnable parameters and no retraining, so it can be dropped onto an existing deployed segmentation model as a post-processing layer. It is deterministic, which matters for reproducible evaluation and for certification-style pipelines, and the test-time augmentation cost is a fixed 13 forward passes rather than an open-ended search.

Future Directions

• Closing the ranking gap. With 67.2% any-candidate coverage but only 20.0% top-1 coverage, better confidence estimation for ordering masks is the clearest remaining headroom, and the paper explicitly motivates joint attention to construction and ranking.
• Learning the decoding decisions. Thresholds (0.15 support, 0.40 core), neighborhood size (32), radius limits, and the class-conditioned route are currently fixed per class; making these learned or adaptively tuned per scene is a natural extension.
• Extending beyond the nine categories. The problem is formulated for class 0 as background and classes 1–9 as targets; broader vocabularies of interaction elements would test whether the graph-geodesic partitioning generalizes.
• Reducing the augmentation cost. Thirteen forward passes are affordable but not free at scan scale; whether fewer, better-chosen transforms or a cheaper consensus scheme preserves the 12-distinct-transform benefit is untested here.

Target Audience

Researchers and engineers working on 3D instance segmentation, point-cloud part segmentation, and embodied or robotic perception, particularly those who already have a point-wise predictor and need benchmark-ready ranked instance masks. It is also useful for challenge participants who want a concrete example of how test-time augmentation, threshold-based core/support splitting, and graph-geodesic partitioning fit together, and for practitioners who need a deterministic, training-free post-processing layer. Readers seeking new network architectures or learned grouping modules will not find them here; the contribution is entirely in the decoding pipeline.

Authors’ abstract

Functional segmentation requires locating small interactive parts, such as knobs, buttons, and sockets, in indoor point clouds and returning confidence-ranked instance masks. A point-wise semantic predictor produces a continuous probability field, whereas the benchmark evaluates a discrete set of scored instances. This creates a substantial field-to-instance gap. We present TopoField, a deterministic decoder built on top of VOLT. TopoField first converts 13 test-time forward passes into a semantic consensus field, then decodes instance candidates from the consensus geometry, and finally ranks the candidates in a class-aware manner. The augmentation stage retains the processed-point identity of every fragment prediction, accumulates probabilities in a common point array, normalizes by coverage count, and fuses the resulting caches. The decoding stage separates high-confidence cores from broader support regions, constructs a symmetric radius-pruned point graph, and uses a multi-source graph-geodesic partition to split support regions that contain multiple seeds. On 561 ground-truth instances across 30 validation scenes, an IoU-0.5-valid candidate exists for 67.2% of instances, and top-1 coverage is 20.0%. These complementary measurements motivate joint attention to candidate construction and ranking. TopoField achieves 28.286% AP, 41.383% AP50, and 47.558% AP25. TopoField achieved second place on the ECCV 2026 SceneFun3D Challenge.