GROVER: Graph-guided Representation of Omics and Vision with Expert Regulation for Adaptive Spatial Multi-omics Fusion

AI summary

Overview

Research area: Multimodal spatial omics integration — combining spatially resolved transcriptomics, proteomics, and histopathology images into a single representation for tissue analysis. This sits at the intersection of computer vision, graph machine learning, and computational biology.

Technical level: Advanced. The paper assumes familiarity with graph neural networks, contrastive learning, mixture-of-experts routing, and spatial omics data formats.

Scope: GROVER is a framework that fuses RNA, protein, and histology-image modalities at single-spot resolution using KAN-based graph encoders, masked contrastive alignment, and a self-adaptive mixture-of-experts, and is evaluated against three baselines on four public 10x Visium datasets.

What This Paper Is About

Spatial omics technologies measure gene expression, protein abundance, and (in some cases) epigenomic state at spots across a tissue section, but they miss the morphological context that a histopathology image provides. The paper's goal is to build a single unified representation per spot that combines transcriptomic, proteomic, and image information, even though these modalities differ drastically in semantics, sit at different resolutions (high-resolution images vs. lower-resolution sequencing spots), and are each corrupted by different technical and biological noise. GROVER addresses this by learning modality-specific graph embeddings, aligning them with contrastive learning, and then adaptively deciding per spot which modalities deserve to be trusted.

Key Contributions

1. A spot-feature-pair contrastive learning strategy that bridges the semantic gap between omics data and histological images in a weakly paired setting, using a similarity mask to suppress false negatives caused by biologically similar spots.
2. A modality-specific multi-expert architecture with gated routing that assigns adaptive weights to RNA, ADT (protein), and image experts at each individual spot, filtering unreliable signals with a confidence threshold.
3. A hybrid graph encoder (KAN-GCN) — a Graph Convolutional Network whose fixed linear transformations are replaced by Kolmogorov–Arnold Network univariate functions, giving kernel-based nonlinear message passing over both spatial and feature-similarity graphs.
4. A complete adaptive fusion pipeline combining attention-weighted within-modality fusion, cross-modality contrastive alignment, and reconstruction-based training, validated on four real-world spatial multi-omics datasets.

Main Findings

• GROVER leads on nine clustering metrics across four datasets. The benchmark covers ARI, NMI, FMI, SC, AMI, Jaccard, DBI, CHI, and Purity, on human tonsil, breast cancer, glioblastoma, and tonsil with add-on antibodies datasets. GROVER consistently ranks among the top-performing methods.
• Reported improvements over baselines. The paper states GROVER improves ARI by 4.4% and SC by 32.8% on the Human Tonsil dataset, and boosts ARI and FMI by 2.6 and 3.3 percentage points on Human Breast Cancer. On Human Tonsil, GROVER reaches ARI 45.2±7.8, NMI 54.3±9.9, and CHI 2494.4±285.5.
• A bimodal method can beat a trimodal one. SpatialGlue (transcriptomics + proteomics) often outperforms MISO (which also uses images), which the authors interpret as evidence that uniformly fusing all modalities is not ideal — motivating GROVER's adaptive weighting.
• Expert routing matters. Removing the MoE routing and replacing it with simple summation dropped ARI, NMI, and FMI by 4.0%, 2.2%, and 3.7% respectively on the Human Tonsil with add-on antibodies dataset.
• Contrastive alignment preserves spatial coherence. Removing the spot-feature-pair contrastive loss reduced SC by 16.6%.
• KAN-GCN trades unsupervised for supervised performance. Replacing KAN-GCN with a standard GCN slightly improved unsupervised metrics SC and DBI (SC 52.6±1.1 vs. 38.2±1.2; DBI 89.6±4.7 vs. 105.8±2.9) but clearly decreased supervised metrics.
• Robustness to the gating threshold. Performance is stable across a range of confidence thresholds γ; γ=0.1 is suboptimal, performance peaks around γ=0.2 and 0.3, and declines slightly at higher values due to over-reliance on a single modality.
• Robustness to the contrastive loss weight. Minimal variation in metrics was observed across values of λ, which the authors set to 2.
• Qualitative gains. On Human Tonsil, GROVER better reconstructs germinal center (GC) regions and boundaries; SpatialGlue also detects GC regions but often splits the same type into subgroups.

Methodology in Plain English

The pipeline has four stages.

1. Build two graphs per modality. For each spot, the authors construct a spatial graph from physical (x, y) coordinates using K-Nearest Neighbors, and a separate feature graph from the modality's own feature vectors (also KNN). The spatial graph captures "what is near what"; the feature graph captures "what looks like what."

2. Encode with KAN-GCN. Rather than the standard GCN update, which multiplies node features by a learned weight matrix, GROVER replaces that matrix with a set of trainable univariate functions inspired by Kolmogorov–Arnold Networks. This makes the message-passing nonlinear and more expressive. The spatial graph and the modality-specific feature graph are encoded separately, then combined per spot through an attention layer that learns how much to trust the spatial versus the feature view.

3. Align modalities with masked contrastive learning. Image patches are extracted at each spot's coordinates and encoded by a pretrained vision foundation model. The authors then compute cosine similarity matrices between spots within a modality, and whenever similarity exceeds a threshold δ, they mask out that pair from the contrastive denominator — preventing biologically similar but non-identical spots from being treated as negatives. A masked InfoNCE loss is applied symmetrically to three modality pairs: RNA–ADT, RNA–Image, and ADT–Image.

4. Route through experts and reconstruct. The three aligned embeddings are averaged into a gating input, and a small gating network produces three confidence scores (one per modality) via softmax. Scores below a threshold γ are zeroed out, the rest are renormalized, and each modality's dedicated feed-forward expert is weighted accordingly. If all scores fall below the threshold, the model falls back to the single highest-confidence expert. Training combines per-modality reconstruction losses (decoded through the spatial adjacency) with the contrastive losses, balanced by λ.

Evaluation uses cell-type clustering labels derived separately from RNA and ADT, with mean and standard deviation reported across five clustering settings (10 to 6). Experiments ran on dual NVIDIA RTX A5000 GPUs (24 GB) and dual Intel Xeon Silver 4210R CPUs (2.40 GHz, 20 cores × 2); GROVER converged within 300 epochs. Code is available at https://github.com/Xubin-s-Lab/GROVER.

Why This Matters

Spatial transcriptomics and spatial proteomics were named Methods of the Year by Nature in 2021 and 2024 respectively, so the field is moving fast — but integration methods have lagged. Most prior work, including SpatialGlue and COSMOS, fuses only molecular modalities and ignores the histological image; only MISO brings images in, and it treats every modality equally at every spot. GROVER's core insight is that modality quality varies spot by spot, so the fusion itself should be adaptive. This reframes multimodal integration as a per-location reliability problem rather than a fixed-weight averaging problem.

Real-world applications:

• Tumor microenvironment characterization — mapping which cell populations sit where inside a tumor, using RNA, protein, and morphology together.
• Spatial domain identification — segmenting a tissue section into biologically coherent regions (the paper's own qualitative example recovers germinal center boundaries in tonsil).
• Cell type annotation — resolving spot types where a single modality is ambiguous, particularly where sequencing dropout obscures the signal.
• Disease tissue analysis — the abstract frames the work as enabling "comprehensive disease tissue analysis" across molecular and morphological layers.

Industry relevance: The framework is intentionally modular — the authors note it can accept any state-of-the-art pathology foundation model in place of the one they used, and any omics preprocessing. That makes it attractive to spatial omics platform vendors, computational pathology groups, and pharma teams running tissue-based target discovery, where pairing cheap H&E histology with expensive molecular assays is a routine workflow.

Future Directions

• Extending beyond three modalities. The abstract explicitly names epigenomics as part of the multimodal spatial omics paradigm, but the implementation and experiments cover RNA, ADT, and image only. Whether the gating and contrastive machinery scales to four or more experts is untested.
• Benchmarking against stronger ground truth. The authors acknowledge the absence of ground-truth annotations spanning all modalities and instead evaluate against RNA-derived and ADT-derived labels separately, which they credit for the higher variance some methods show. A unified multimodal ground truth would give cleaner comparisons.
• Resolving the KAN-GCN trade-off. Swapping KAN-GCN for a standard GCN improved SC and DBI while hurting supervised metrics. Understanding why nonlinear message passing helps label-based clustering but hurts geometry-based metrics is an open question.
• Adapting routing to more pathology foundation models. Since the framework is modular, testing how sensitive the MoE routing and contrastive alignment are to the choice of vision encoder would clarify how much of GROVER's gain comes from the image representation versus the fusion design.

Target Audience

Computational biologists and bioinformaticians working on spatial omics who need a practical integration tool; machine learning researchers interested in mixture-of-experts routing, contrastive alignment under weak pairing, or Kolmogorov–Arnold Network variants applied to graphs; and computational pathology researchers who want to incorporate histology images into molecular tissue analysis. A reader without a background in graph neural networks or omics data structures will find the methods section dense, since it is written in full mathematical notation with little introductory scaffolding.

Authors’ abstract

Effectively modeling multimodal spatial omics data is critical for understanding tissue complexity and underlying biological mechanisms. While spatial transcriptomics, proteomics, and epigenomics capture molecular features, they lack pathological morphological context. Integrating these omics with histopathological images is therefore essential for comprehensive disease tissue analysis. However, substantial heterogeneity across omics, imaging, and spatial modalities poses significant challenges. Naive fusion of semantically distinct sources often leads to ambiguous representations. Additionally, the resolution mismatch between high-resolution histology images and lower-resolution sequencing spots complicates spatial alignment. Biological perturbations during sample preparation further distort modality-specific signals, hindering accurate integration. To address these challenges, we propose Graph-guided Representation of Omics and Vision with Expert Regulation for Adaptive Spatial Multi-omics Fusion (GROVER), a novel framework for adaptive integration of spatial multi-omics data. GROVER leverages a Graph Convolutional Network encoder based on Kolmogorov-Arnold Networks to capture the nonlinear dependencies between each modality and its associated spatial structure, thereby producing expressive, modality-specific embeddings. To align these representations, we introduce a spot-feature-pair contrastive learning strategy that explicitly optimizes the correspondence across modalities at each spot. Furthermore, we design a dynamic expert routing mechanism that adaptively selects informative modalities for each spot while suppressing noisy or low-quality inputs. Experiments on real-world spatial omics datasets demonstrate that GROVER outperforms state-of-the-art baselines, providing a robust and reliable solution for multimodal integration.