Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy

AI summary

Overview

Research area: Medical computer vision — self-supervised 3D–2D deformable registration for augmented-reality-guided laparoscopic liver surgery.

Technical level: Advanced. The paper combines rigid point-cloud matching (GeoTransformer, Sinkhorn, PROSAC, trimmed ICP), implicit neural deformation fields with SIREN activations, and differentiable point rasterization for mask-gated self-supervision.

Scope: The paper proposes Vis2Reg, a landmark-free, geometry-only framework that aligns a complete preoperative 3D liver model to partial, view-dependent intraoperative laparoscopic observations using visibility-aware self-supervision, and evaluates it on the public P2I-LReg benchmark.

What This Paper Is About

During laparoscopic liver surgery, surgeons see only a narrow, partially occluded view of the liver surface, with no direct sight of subsurface tumour–vessel relationships. Augmented reality can help by overlaying a patient-specific preoperative 3D liver model onto the live video, but this requires accurately aligning (registering) the 3D model to what the camera actually sees — a problem made hard by large non-rigid deformation, occlusion, specularities, smoke, blood, and the fact that no 3D ground-truth registration exists for real surgery. Vis2Reg addresses this by learning deformation using only geometry that is genuinely visible in the intraoperative masks, rather than penalising the model for unobserved anatomy.

Key Contributions

1. Visibility-aware self-supervision formulation. The authors construct a mask-consistent visible-domain 3D supervision signal for landmark-free laparoscopic liver registration, restricting supervision to geometrically observable regions and thereby avoiding penalties on unobserved model regions.
2. Differentiable rasterization and mask-guided back-projection mechanism. A differentiable point rasterizer renders depth and silhouette per view from the warped cloud, and mask-gated back-projection converts mask-consistent visible pixels into an explicit 3D supervision set.
3. Integration of robust rigid initialization with an implicit neural deformation field. A rigid seed estimated via soft correspondence (GeoTransformer contextualization, Sinkhorn-normalized soft assignment, mutual nearest neighbors), PROSAC hypothesis generation, and trimmed point-to-point and point-to-plane ICP is combined with a SIREN-based MLP deformation field, yielding improved robustness and near-real-time performance on in-vivo laparoscopic datasets.

Main Findings

• Rigid initialization on synthetic data: Vis2Reg achieves RRE of 0.08° and RTE of 0.26 mm, better than Self-P2IR (RRE 0.21°, RTE 1.32 mm) and substantially better than the standalone GeoTransformer matcher entry (RRE 31.30°, RTE 78.10 mm). The paper attributes the GeoTransformer gap to its standalone use with built-in closed-form pose estimation, no robust outlier rejection (e.g., RANSAC), and no ICP refinement.
• Real intraoperative registration: Vis2Reg reaches Dice of 92.60 ± 7.24 % and Chamfer Distance of 1.43 ± 1.26 mm on the real benchmark, the best mean performance among compared methods.
• Comparison against the strongest prior method: Relative to Self-P2IR, Vis2Reg improves Dice by 13.71 percentage points and reduces CD by 1.54 mm. Because Self-P2IR also starts from a near-perfect rigid initialization (RRE 0.21°, RTE 1.32 mm), the authors argue the gain stems from visibility-aware non-rigid supervision rather than initialization.
• Baseline results on real data: GeoTransformer reaches Dice 71.16 ± 6.46 % and CD 3.43 ± 0.96 mm; DPF reaches Dice 72.19 ± 5.97 % and CD 3.31 ± 1.18 mm; PointSetReg reaches Dice 75.24 ± 5.92 % and CD 3.20 ± 1.03 mm; Self-P2IR reaches Dice 78.89 ± 6.76 % and CD 2.97 ± 1.06 mm. DPF and PointSetReg are non-rigid models and are marked "–" in the synthetic rigid block.
• Ablation on removing visibility supervision: Without the visibility-aware term, Dice falls to 74.29 % and CD rises to 3.12 mm.
• Ablation on mask-gating: Removing mask-gating (using all pixels with rendered depth > 0 rather than intersecting with the mask) gives Dice 79.57 % and CD 3.04 mm.
• Ablation on the one-way Chamfer: Replacing the one-way observation-to-model Chamfer with symmetric Chamfer gives Dice 84.48 % and CD 2.98 mm.
• Ablation on rigid initialization: Weakening the rigid seed causes the largest single drop, to Dice 69.32 % and CD 3.64 mm.
• Summary of supervision ablations: Removing visibility-aware supervision or mask-gating alone reduces Dice by 13 to 18 percentage points.
• Efficiency: Vis2Reg runs at 111.38 ms per forward pass using 1.95 GB of GPU memory on an NVIDIA L40S. The paper notes this covers forward registration after the intraoperative clouds and masks are available, and excludes depth and mask reconstruction.
• Metric interpretation: Dice is described as the primary AR-overlay metric, measuring overlap between the rasterized registered silhouette and the liver mask, while CD is a complementary surface-distance measure and explicitly not a target registration error.

Methodology in Plain English

The framework treats the problem as partial-to-complete 3D registration: a complete preoperative liver point cloud must be aligned to a sparse, view-dependent intraoperative point cloud reconstructed from laparoscopic images.

1. Reconstructing the intraoperative view. Each of F = 3 views (a short, non-temporal local window corresponding to a near-static liver) is turned into a point cloud using monocular depth from DepthAnything, the liver mask, and back-projection with known camera intrinsics. Masks and intrinsics are used only as supervision signals, not as model inputs at inference.
2. Encoding geometry. A multi-layer EdgeConv encoder captures local shape context through dynamic k-nearest-neighbor aggregation. A Siamese matching encoder extracts features for rigid alignment from the preoperative and intraoperative clouds, while a separate registration encoder extracts deformation features conditioned on the pair.
3. Getting a stable rigid starting pose. Contextualized features form a similarity matrix, Sinkhorn normalization produces a soft assignment with confidence weights, and mutual nearest neighbors build a correspondence set. PROSAC generates pose hypotheses from minimal samples, ranked by weighted inlier scores; the top hypotheses are refined with trimmed point-to-point and point-to-plane ICP, and the best provides the rigid initialization.
4. Modeling deformation. Non-rigid deformation is a continuous implicit displacement field. Each source point's positional encoding is concatenated with its pair-conditioned registration feature and passed to a SIREN-activated MLP, which predicts a 3D displacement. The rigid transform is then applied to the deformed point to produce the warped cloud. Because both the feature and the rigid transform depend on the intraoperative input, deformation adapts to each case rather than being a fixed function of the preoperative model.
5. Building honest supervision. The warped cloud is rendered per view by differentiable point rasterization into depth and silhouette. A pixel is treated as visible only if rendered depth is greater than zero and the intraoperative mask equals one; those pixels are back-projected analytically into an explicit visible-domain 3D supervision set. This supervision is mask-gated and one-way (observation-to-model), unlike the symmetric rendered-mask consistency of Self-P2IR, so unobserved model regions are never penalized.
6. Training objectives. The total loss is a weighted sum of an observation-to-model one-way Chamfer term, a symmetric visible-domain term aligning the mask-gated supervision set with the intraoperative cloud, a silhouette term combining BCE and Dice loss against the mask, plus regularization on deformation magnitude, local smoothness, and local distance (topology) preservation. Loss weights are λ3D = 0.5, λvis = 1.2, λsil = 1.0 (BCE and Dice weights both 1.0), λdef = 0.1, λsmooth = 0.1, and λtopo = 0.3.
7. Two-stage schedule. The matching and rigid-seed modules are pretrained on synthetic data with ground-truth poses (non-rigid field disabled), then trained on real data with a short rigid warm-up followed by joint visibility-aware optimization. Stage-2 real-data training runs 60 epochs at batch size 1, with a 20-epoch rigid warm-up during which the non-rigid branch and deformation regularizers are frozen.
8. Data and setup. Evaluation uses the public P2I-LReg dataset (346 keyframes from 21 patients), split 12/4/5 across training/validation/testing following the official patient-level protocol and averaged over 5-fold cross-validation. Rigid initialization additionally uses a patient-specific Landmark-Free Synthetic Dataset rendered from preoperative models via physics-based Blender simulation with diverse viewpoints and occlusions (2500 samples per patient; 60%/20%/20% split; training-fold patients only). Inputs are at real-world scale with no centering or normalization; points are resampled or zero-padded to N_max = 6000. Optimizer is AdamW (lr = 3×10⁻⁴, wd = 10⁻⁴) with cosine scheduling and automatic mixed precision. Rasterization uses 16 points per pixel and a maximum of 5000 visible points.

Why This Matters

Impact on research. The paper reframes laparoscopic registration supervision around what is actually observable. By showing that mask-gated, one-way, visible-domain supervision outperforms symmetric rendered-mask consistency (Dice 92.60 % versus 84.48 % when symmetric Chamfer is used), it provides evidence that how visibility is modeled — not just whether self-supervision is used — drives registration quality. It also shows that a strong rigid seed matters enormously even when non-rigid learning is good, since weakening the seed produced the largest single ablation drop.

Real-world applications:

• Augmented-reality overlay in laparoscopic and robotic liver resection, where preoperative tumour and vessel models are projected onto the live video to restore spatial awareness beyond the visible surface.
• Surgical planning and intraoperative guidance for assessing tumour–vessel relationships that are otherwise hidden from the camera.
• Any monocular-camera surgical registration setting with partial visibility and occlusion, since the method requires no 3D ground truth and, at inference, no masks or explicit correspondences.
• Point-cloud registration pipelines more broadly where supervision signals are view-dependent and incomplete.

Industry relevance. A 111.38 ms forward pass on the L40S GPU with 1.95 GB of memory places the method in near-real-time territory for intraoperative use, which matters for surgical robotics and AR navigation vendors. The approach works from geometric inputs alone at inference and requires no 3D ground-truth registration labels for training, lowering the annotation barrier for commercialization. The authors disclose funding from the Engineering and Physical Sciences Research Council (Grant No. UKRI914) and declare no competing interests.

Future Directions

• Extending beyond surface geometry. The authors explicitly state that evaluating tumour and vascular structures would require internal anatomical ground truth that is unavailable here, and leave this to future work.
• Generalizing beyond the P2I-LReg benchmark. The paper restricts its evaluation and claims to P2I-LReg, so external validation on other in-vivo laparoscopic datasets remains open.
• Handling temporal dynamics. The current formulation uses F = 3 as a short, non-temporal window corresponding to a near-static liver; scenarios with substantial motion during the window are not addressed.
• Strengthening occlusion-robust correspondence. The paper notes that generic low-overlap point-cloud registration methods such as PREDATOR and GeoTransformer improve correspondence robustness but are not tailored to visibility-limited surgical self-supervision, suggesting room to design matching tailored to this regime.

Target Audience

Researchers and practitioners in medical image computing and surgical computer vision, particularly those working on deformable registration, augmented-reality-guided surgery, and self-supervised learning under partial observation. It is also relevant to surgical robotics and AR navigation engineers who need registration accuracy with near-real-time latency and no 3D ground-truth labels. Readers should be comfortable with point-cloud learning, implicit neural fields, and differentiable rendering; the methods section is not beginner-level.

Authors’ abstract

Accurate 3D--2D liver registration, which aligns preoperative 3D models to partial, view-dependent intraoperative surface observations, is critical for AR-guided laparoscopic surgery but remains challenging due to severe occlusion, limited visibility, and the lack of 3D ground-truth supervision. Existing landmark-free approaches perform partial-to-complete geometric alignment, yet robust self-supervision under extreme partial visibility remains difficult. We propose Vis2Reg, a visibility-aware registration framework that explicitly constrains deformation using mask-consistent visible regions. We introduce a visibility-aware self-supervision that derives a visible-domain 3D supervision signal from intraoperative masks, enabled by differentiable point rasterization and mask-guided back-projection. This formulation improves robustness under severe occlusion while maintaining fully self-supervised learning. Vis2Reg combines a robust geometric rigid initialization module with an implicit neural deformation field for stable alignment. Vis2Reg achieves a Dice score of 92.6\% and a Chamfer Distance of 1.43 mm on real intraoperative datasets, with 111 ms per-frame inference time, demonstrating both accuracy and practical efficiency.