MDND: Unsupervised Learning Guided by Non-Differentiable Refinement for Shape Correspondence

AI summary

Overview

• Research area: Non-rigid 3D shape correspondence, deep functional maps (DFM), spectral geometry, unsupervised geometric deep learning.
• Technical level: Advanced. The paper assumes familiarity with the functional map framework, Laplace-Beltrami eigenbases, spectral wavelets, and differentiable map solvers.
• Scope: The paper proposes MDND, a dual-branch deep functional map architecture in which a non-differentiable, hybrid-basis iterative refiner supervises a differentiable feature-learning branch, and evaluates it against a broad set of axiomatic, supervised, and unsupervised baselines.

What This Paper Is About

Deep functional map pipelines for shape correspondence are built on end-to-end differentiability, which prevents them from incorporating powerful axiomatic refinement algorithms that contain non-differentiable operations (nearest-neighbor searches, discrete steps). This caps their accuracy, particularly on non-isometric deformations and shapes with topological noise. The paper removes that constraint by treating a non-differentiable refinement algorithm as a supervisory oracle: a differentiable branch learns features whose soft correspondences are pulled toward the high-quality maps produced by the non-differentiable branch, using a single consistency loss.

Key Contributions

1. MDND, a merging-differentiable-and-non-differentiable paradigm. The authors state this is the first approach to integrate non-differentiable iterative refinement into the deep functional map framework, so that gradients for the feature extractor propagate only through a parallel differentiable branch while the refinement branch acts purely as a supervision source.
2. Hybrid Wavelet Filtering (HWF), a new refinement oracle. HWF generalizes the principles of the MWP refiner (Hu et al. 2021) from the LBO eigenbasis to a hybrid basis combining the intrinsic LBO basis with an extrinsic basis derived from an elastic thin-shell energy (ELA, Hartwig et al. 2023). The paper states it presents a different theoretical derivation from MWP, provided in the appendix.
3. A streamlined unsupervised training objective. Instead of balancing multiple structural regularizers (orthogonality, bijectivity), MDND uses a single alignment loss that enforces consistency between the soft pointwise map of the differentiable branch and the refined functional maps from the non-differentiable oracle, which are treated as detached constants during backpropagation.
4. State-of-the-art results across near-isometric, non-isometric, and topologically noisy benchmarks, with the largest stated margins on non-isometric and topologically noisy shapes.

Main Findings

• A non-differentiable oracle beats a differentiable solver. In the first ablation (Table 2), replacing the non-differentiable refinement branch with a standard differentiable solver that computes the functional map directly from features degrades mean geodesic error (×100) from 1.9 to 2.0 on SCAPE, from 3.1 to 4.2 on SMAL, and from 3.5 to 9.4 on TOPKIDS.
• The hybrid basis is the better oracle. In the second ablation (Table 3), MWP using only the LBO basis yields 2.2 (SCAPE), 4.9 (SMAL), 14.7 (TOPKIDS); MWP using only the ELA basis yields 2.3, 4.1, 5.5; and HWF using LBO + ELA yields the best result on all three, at 1.9, 3.1, 3.5.
• Topological noise is where the gain is largest. On the TOPKIDS dataset, the paper reports the best performance of any compared method, improving on the next-best approach by a stated 30%.
• Near-isometric performance is comparable, generalization is strong. On the standard remeshed FAUST and SCAPE test sets the method is reported as comparable to the state of the art, while the paper highlights its performance in the F→S and S→F generalization tests as strong.
• Large margins on non-isometric shapes. On SMAL, which contains various tetrapod species, the paper reports clear outperformance of approaches relying solely on the LBO basis; on DT4D-H it reports superiority over most competitors in both intra-class and inter-class matching.
• Robustness to cross-dataset transfer. Figure 3 visualizes texture transfer when training and testing on different datasets.
• A simplified training landscape. Using one consistency loss avoids the weight-balancing problem of competing orthogonality and bijectivity terms.

Methodology in Plain English

The framework has two branches that share one feature extractor.

The learning branch. A Siamese network with shared weights — DiffusionNet, chosen because it is robust to discretization and handles bilateral symmetry — extracts per-vertex features for the source and target shapes. A soft pointwise correspondence is produced by a temperature-scaled Softmax over the feature similarity matrix. This branch is fully differentiable, so gradients can flow through it.

The refinement branch. From the same learned features, the method computes a hard pointwise map with a nearest-neighbor search, stored memory-efficiently as a single vector of indices. That hard map is the input to Hybrid Wavelet Filtering, a parameter-free iterative loop:

1. convert the hard pointwise map into functional maps in both the LBO and ELA bases;
2. apply multiscale spectral manifold wavelet filters to both functional maps;
3. convert the filtered maps back into an updated hard pointwise map by nearest-neighbor search in the combined spectral embedding;
4. repeat for a fixed number of iterations.

Because this loop contains non-differentiable operations, no gradients are tracked through it.

Tying them together. The refined functional map from the oracle becomes a fixed target in a single alignment loss, which measures how far the embedded soft pointwise map from the learning branch is from the embedded oracle map. Minimizing this loss teaches the feature extractor to produce initializations that the refiner can push to high accuracy. The intuition is that the network never has to learn the difficult spectral optimization itself — it only has to learn features that make that optimization easy.

Experimental setup. The implementation is in PyTorch, run on a single NVIDIA RTX 4090 GPU. All methods and datasets use 128-dimensional HKS descriptors as input features. The LBO basis size is 128 and the ELA basis size is 200 for all datasets, except SMAL where it is reduced to 100. Errors are reported as mean geodesic error normalized by the square root of the source shape's area and multiplied by 100 for readability. No post-processing or test-time adaptation is applied. The code is released at https://github.com/AMAWDBAC/MDND.

Why This Matters

Impact on research. The dominant DFM design assumes that every stage must be differentiable, which forces axiomatic refinement methods to be reimplemented as approximations — differentiable versions that often require storing dense soft-correspondence matrices with quadratic memory cost, or differentiating through numerically unstable linear solvers. This paper argues that this premise is misplaced: the label-free but exact axiomatic algorithm can instead serve as a target that the network learns to imitate. If the approach generalizes, it opens a route for any strong non-differentiable classical algorithm to supervise a neural network, which is a broadly applicable idea beyond shape matching.

Real-world applications (as listed or implied by the paper's own motivation):

• Texture transfer between deformable 3D models.
• Shape interpolation and morphing between deformations.
• Statistical shape analysis and morphable model construction.
• Character rigging and animation transfer across scanned or artist-modeled meshes.

The topologically noisy case matters especially because real scans often have spurious handles, holes, and boundary artifacts that break clean isometric assumptions.

Industry relevance. Areas that depend on transferring information between meshes with different topology or deformation — game and film asset pipelines, medical shape analysis, 3D garment and pose fitting — benefit from correspondences that survive non-isometric deformation and mesh defects. The method is also memory-conscious: the hard map is stored as an index vector rather than a dense soft matrix, and the whole pipeline runs on a single consumer-grade GPU, which matters for practical deployment. The paper does not report runtime measurements or inference costs.

Future Directions

• Bridging spectral and spatial domains. The authors state as an acknowledged limitation that the approach is purely spectral, and propose integrating explicit spatial deformation models as future work for greater robustness.
• Making the oracle component-agnostic. The framework is decoupled from any specific refiner — the ablations already swap MWP variants for HWF — so the natural question is whether other non-differentiable solvers (partial matching, symmetry handling, multi-shape consistency) can be plugged in as oracles.
• Extending beyond the reported benchmarks. The paper evaluates on FAUST, SCAPE, SMAL, DT4D-H, and TOPKIDS, with a BCICP column present in Table 1; the provided content does not give dataset sizes, training set sizes, iteration counts, or runtime, and the appendix derivation is truncated, so the theoretical justification cited for HWF is not fully verifiable from the supplied text.
• Scalability to high-resolution meshes. Since one motivation for rejecting differentiable axiomatics was quadratic memory cost, establishing how MDND scales in mesh resolution and how the LBO/ELA basis sizes (128 and 200, and 100 on SMAL) should be chosen would be a useful follow-up.

Target Audience

Researchers and graduate students working on non-rigid shape matching, functional maps, spectral geometry, or geometric deep learning, particularly those interested in hybridizing classical optimization with learned representations. Practitioners in 3D graphics, animation, and shape analysis who need correspondences robust to non-isometric deformation and topological noise will find the practical results and the released code most relevant. Readers without a background in Laplace-Beltrami eigenbases, functional maps, and spectral wavelet filtering will need to consult the cited foundational work first.

Authors’ abstract

Deep functional map frameworks (DFM) for shape correspondence are powerful, yet fundamentally limited by their reliance on end-to-end differentiability. This constraint prevents the integration of highly accurate, non-differentiable refinement techniques, capping their overall performance, especially on challenging non-isometric shapes. To overcome this, we introduce MDND, a novel DFM paradigm built on the principle of merging differentiable and non-differentiable components. Our framework facilitates unsupervised learning guided by an internal, non-differentiable refinement. Specifically, MDND employs a dual-branch architecture: a non-differentiable refinement branch leverages a novel, multiscale iterative solver to produce highly robust correspondences, acting as a refined target. Concurrently, a fully differentiable branch learns to predict correspondences from features. The entire system is trained end-to-end without supervision by enforcing a consistency loss that compels the differentiable branch to learn from the superior, refined results of the non-differentiable branch. Extensive experiments show that MDND sets a new state-of-the-art, demonstrating remarkable robustness on shapes with non-isometric deformations and topological noise.