How Geometry Shapes Generative Model Evaluation on Riemannian Manifolds

AI summary

Overview

Research area: Machine learning — evaluation of generative models whose data live on curved spaces (Riemannian manifolds), combining optimal transport, kernel two-sample testing, and Riemannian geometry.

Technical level: Intermediate. The paper is readable with a working knowledge of generative modelling and basic differential geometry; the core ideas are explained through intuition about geodesics and volume growth rather than heavy formalism.

Scope (one sentence): The paper systematically compares six variants of two-sample evaluation metrics — 1-Wasserstein distance and squared MMD, each instantiated with geodesic, ambient Euclidean, or tangent-space distances — across spherical, flat, and hyperbolic spaces of varying curvature and dimension, and reports how geometry changes both the measured discrepancy and the metric's ability to detect sample degradation.

What This Paper Is About

Evaluating a generative model usually means measuring a distance between generated samples and target samples, and on a curved manifold there is no single natural way to do this: one can use the geodesic distance on the manifold, the Euclidean distance between the ambient coordinates in which the manifold is embedded, or the Euclidean distance after mapping points to a shared tangent space. The paper asks how these three choices, together with the sign and magnitude of curvature and the dimension of the space, affect what an evaluation metric reports and how reliably it detects worsening sample quality. The goal is to give practitioners concrete guidance on which evaluation distance to use, when conclusions transfer between metrics, and when a metric's apparent ranking should not be trusted.

Key Contributions

1. A controlled comparison framework for evaluation geometry. The authors construct a unified setup in which spherical, Euclidean, and hyperbolic spaces are parameterised in geodesic polar coordinates around a fixed origin, so that only curvature and dimension change between conditions. Distributions are placed on the manifold via the exponential map from tangent-space samples rescaled by the curvature radius, keeping distributions at comparable relative scale across curvatures so that differences are not merely artefacts of the same tangent samples occupying different fractions of the manifold.

2. A six-way decomposition of two standard metrics. Wasserstein-1 and squared MMD are each computed under three evaluation distances (geodesic, ambient, tangent), and the paper measures two properties: agreement (whether metrics produce the same ranking of generated sample sets) and sensitivity (whether a metric's value grows with the size of a perturbation, measured as a signal-to-noise ratio).

3. A diagnostic link between geometry and metric reliability. The paper defines the Jacobian determinant of the exponential map, J_κ(r) = (sn_κ(r)/r)^(d-1), together with its distributional mean μ_Jκ(p) = E_{x~p}[J_κ(r(x))], as a measure of curvature-induced volume scaling, and correlates this quantity with how much reliability geodesic evaluation gains over ambient or tangent evaluation.

4. Practical reporting recommendations (Table 2), covering how to report evaluation distances, when to cross-check with a second distance, when to use more than one metric family, and when to run a controlled-noise sanity check before trusting MMD in high dimensions.

Main Findings

• No metric is uniformly reliable. Neither W_1 nor MMD² is consistently the better or more trustworthy evaluation across geometries; the paper states that both metric family and evaluation distance can change measured discrepancies and noise sensitivity.

• Changing distance within a metric moves values but rarely rankings. The appendix results are described as showing that rankings are relatively stable across the three evaluation distances within a single metric type, but weak and target-dependent across metric families (i.e., W_1 versus MMD²).

• Dimension does not uniformly amplify geometric error. In hyperbolic space the gap between geodesic and non-intrinsic W_1 decreases with dimension d, whereas geodesic MMD² increasingly diverges from both tangent and ambient evaluation. Spherical spaces show weaker divergence, with tangent and ambient approximations behaving similarly to one another.

• MMD loses sensitivity at high dimension, Wasserstein does not. Under controlled noise, W_1 remains near-monotone across dimensions, whereas MMD² weakens at high d. The paper cites prior work (Reddi et al., 2014) on kernel discrepancies losing sensitivity in high dimensions as a partial explanation. At d = 16, both metrics generally increase with controlled noise, but the response differs across curvature and across evaluation distance.

• Noise-detection reliability is not a property of the metric alone. The global SNR, computed as SNR_D(0, L−1) over the ordered noise levels, varies systematically across the (κ, d) grid and with the choice of evaluation distance, with especially distance-dependent response strength in negatively curved spaces.

• Volume scaling partially explains the reliability gaps. Spearman correlations between log μ_Jκ(p_1) and log₂ of the SNR ratio (geodesic versus alternative) on the Gaussian-to-Gaussian task are: for W_1, 0.63 with the ambient alternative and 0.20 with the tangent alternative; for MMD², 0.62 with both alternatives. The strong ambient correlations support the view that where volume grows rapidly, ambient distances underestimate intrinsic separations. The weak tangent–W_1 correlation suggests that log-mapping preserves enough radial structure that volume scaling alone does not predict the gap.

• Metric-specific computation also matters. Because MMD² averages RBF kernel values with a median-distance bandwidth, curvature-induced distance shifts alter both kernel values and bandwidths, producing positive associations for both the ambient and tangent alternatives; the differing W_1 and MMD² patterns show that volume scaling is not the whole story.

Methodology in Plain English

The authors build a testbed in which the only things that change between runs are curvature and dimension, so that any change in a metric's behaviour can be attributed to geometry rather than to confounds.

Spaces. They use constant-curvature manifolds throughout: positive curvature gives spheres, zero gives Euclidean space, and negative gives hyperbolic space represented with the Poincaré ball model. All spaces are described in geodesic polar coordinates around a fixed origin — each point is specified by its geodesic radius r from the origin and an angular direction. In these coordinates the entire effect of curvature reduces to a single radial scaling function sn_κ(r), which is sin(√κ r)/√κ for positive κ, r for zero, and sinh(√−κ r)/√−κ for negative κ.

Fair placement of distributions. A fixed tangent-space radius means different things at different curvatures, because the natural length scale is the curvature radius R_κ = |κ|^(−1/2). The authors therefore rescale tangent-space samples by 1/√|κ| before pushing them onto the manifold with the exponential map, so a distribution sits at the same relative distance from the origin across curvatures. They show this normalisation cancels the explicit dependence on |κ| in both sn_κ and J_κ while retaining the sign-dependent (sin versus sinh) volume behaviour.

Tasks. Base samples are drawn from a Gaussian with standard deviation 0.3 centred at the origin in the tangent space, and three target families are used: a Gaussian-to-Gaussian baseline (GtG); a Symmetric Gaussian Ring (SGR), a rotationally symmetric shell-like target whose radius can be varied to probe radial volume growth; and a Mixture-of-Gaussians (MoG) target placed symmetrically in the tangent space to test multimodality.

Metrics. W_1 is approximated with entropically regularised Sinkhorn divergence (with debiasing terms to remove regularisation bias). MMD² uses an RBF kernel with bandwidth set to the median non-zero pairwise distance. Each is computed three ways: with geodesic distance d_κ, with ambient Euclidean distance, and with Euclidean distance in the tangent space after log-mapping.

Two kinds of evidence. For sensitivity, they take clean target samples and progressively add Gaussian noise in the tangent space at each sample, then map back to the manifold, producing ordered noise levels. For each pair of consecutive noise levels they compute a signal-to-noise ratio — the mean across resampling replicates of the metric difference divided by its standard deviation — and average these to obtain a "mean consecutive SNR". For realistic error, they instead evaluate samples produced by Riemannian Flow Matching (RFM), where error comes from learned transport rather than injected noise.

The volume diagnostic. To connect geometry to metric behaviour they compute the ratio of the manifold volume element to the flat tangent-space volume element, which is exactly the Jacobian determinant of the exponential map, and average it over the target distribution. Correlating this with the geodesic-versus-alternative SNR gap tests whether volume growth explains the reliability differences.

Why This Matters

Impact on research. Benchmarks in geometric deep learning routinely report a single number — often an MMD or Wasserstein value — without stating which distance was used, and readers assume the number is comparable to numbers reported elsewhere. This paper shows that the same samples evaluated with the same metric family but a different distance convention can yield different discrepancies, and that metrics from different families need not agree on rankings at all. Because the paper finds that rankings are relatively stable across distances within a metric family but weak and target-dependent across families, it suggests the larger risk is comparing across metric families, not across distance conventions.

Real-world applications.

• Molecule and protein design, where RFM and related Riemannian generative models are applied to structured non-Euclidean data (the paper cites Chen & Lipman, 2024 and Zaghen et al., 2026), and where hyperbolic or spherical structure arises in the representation. Choosing MMD in a high-dimensional hyperbolic latent space is exactly the regime where the paper finds sensitivity degrades.
• Spherical Earth and geospatial data, where distributions on the sphere are modelled and evaluated; the paper notes that spherical spaces show weaker divergence between distance conventions, which bounds the practical risk there.
• Hyperbolic image and graph representations, where embeddings are learned in negatively curved space and generative models over those embeddings must be validated.
• Rotation and structured-manifold modelling, cited among the standard application areas of Riemannian generative models.

Industry relevance. Any team deploying a generative model on non-Euclidean data needs a reliable quality signal for model selection and regression testing. The paper's Table 2 gives an operational checklist that is cheap to implement: always report the evaluation distance (and, for MMD, the kernel bandwidth rule); in high-dimensional negatively curved settings, do not rely on ambient-only conclusions but verify with geodesic or tangent evaluation; when comparing models, use at least two metric families; and in high dimensions, run a small controlled-noise check before treating MMD as the primary quality signal.

Future Directions

1. Extending the framework to learned latent manifolds. The authors propose applying the same controlled evaluation to latent spaces whose geometry is itself learned (citing Yue et al., 2026 and Bu et al., 2026), so that the latent geometry and its evaluation metric can be selected jointly — potentially yielding a more sensitive representation rather than one evaluated post hoc.

2. Diagnostics as a design tool, not just a reporting check. The paper frames its noise-sensitivity and volume-scaling diagnostics as inputs to metric selection and representation design; a natural next step is turning them into a procedure that guides architecture and geometry choices during development.

3. Curvature magnitude versus curvature sign. The tangent-space normalisation deliberately removes the explicit dependence on |κ| while retaining the sign-dependent volume behaviour (sin versus sinh). The paper states this controls for global curvature-induced scaling but does not fully normalise curvature effects, leaving open how much of the remaining behaviour is attributable to magnitude versus sign.

4. Reconciling the Wasserstein and MMD patterns. Since the two families respond differently to the same geometric change — and volume scaling predicts the ambient alternative well but the tangent–W_1 case poorly — the mechanism behind metric-specific behaviour in curved space is not fully explained. The paper attributes part of the MMD effect to the interaction between distances and the median-distance bandwidth, but a fuller account is left to future work.

Target Audience

Researchers and practitioners who train or evaluate generative models on non-Euclidean data — geometric deep learning, Riemannian flow matching and diffusion, and manifold-valued data in the sciences. It is most useful to anyone who reports MMD or Wasserstein numbers on spherical, hyperbolic, or otherwise embedded data and needs to know whether those numbers are comparable to others in the literature. It also serves methodologists interested in evaluation metrics for generative models, extending the Euclidean findings of Kansal et al. (2023) — that two-sample criteria differ in sensitivity to controlled failure modes — into the geometric setting. Readers seeking a formal theoretical treatment should note that the paper is empirical and diagnostic in character; the appendix supplies the geometric background, derivations of volume growth in constant curvature, and the full experimental details, with curvature and dimension values listed in Appendix D.

Authors’ abstract

On Riemannian manifolds, generative-model evaluation is geometry-dependent since curvature and dimension jointly determine distances and volume growth. Our framework systematically compares geodesic, ambient, and tangent-space versions of Wasserstein and MMD across different geometries. Using controlled-noise perturbations and Riemannian Flow Matching samples, we study metric agreement, noise sensitivity, and their relation to geometry-induced volume scaling. We find that no metric is uniformly reliable: evaluation distance can affect both measured discrepancies and noise sensitivity, varying across curvature and dimension. Our analysis leads to practical recommendations for reporting evaluation distances, comparing metric types, and calibrating metrics to the geometry of the samples.