From Euclidean to Graph-Structured Data: A Survey of Collaborative Learning

AI summary

Overview

Research area: Distributed and collaborative machine learning — specifically federated learning and decentralized learning — and its extension from Euclidean (grid-like) data such as images and text to graph-structured (non-Euclidean) data. The paper sits at the intersection of three fields: diffusion algorithms on networks, graph representation learning, and collaborative learning.

Technical level: Intermediate. The paper is a survey rather than an experimental paper. It presents formal problem definitions (population loss, empirical loss, parameterized optimization objectives) and assumes familiarity with federated learning, stochastic gradient descent, and graph neural networks, but develops concepts from first principles and includes notation and a glossary in an appendix.

Scope (one sentence): The survey maps the design space of collaborative learning from Euclidean to graph-structured data, organizing it along three dimensions — learning effectiveness, efficiency, and privacy preservation — and consolidating an emerging field that has previously been studied in separate research communities.

Note: this is a survey; it does not report benchmark tables, datasets, or accuracy numbers. The available text contains no experimental results.

What This Paper Is About

Conventional machine learning collects data, trains, and infers in one place, which is limited by scalability and privacy concerns. Collaborative learning (federated and decentralized) lets multiple agents train and infer locally while exchanging only model updates or aggregated information — but almost all of this research targets Euclidean data such as images and text, and misses the relational patterns in applications best represented as graphs. This survey bridges that gap by reviewing foundational collaborative learning for Euclidean data, then extending the discussion to graph-structured data with a taxonomy of graph distribution scenarios, statistical heterogeneities, problem formulations, and algorithmic frameworks.

Key Contributions

1. A unified view across two research areas. The survey maps the design space of collaborative learning from Euclidean to graph-structured data, showing how different approaches relate to each other, rather than treating federated learning on Euclidean data and federated/decentralized graph learning as separate literatures. The authors describe the goal as bridging established techniques for Euclidean data with emerging methods for graph learning.

2. A systematic organization along three core dimensions. Euclidean collaborative learning is reviewed and organized along learning effectiveness, efficiency, and privacy preservation. The same three-dimensional structure is then followed for the graph case, so that results and open questions are comparable across the two settings.

3. A taxonomy of graph distribution scenarios and their associated statistical heterogeneities. The survey introduces a taxonomy for how graph data (or information) can be distributed, characterizes the associated statistical heterogeneities, and develops standardized problem formulations and algorithmic frameworks. The authors note that the ways data or information can be distributed are significantly more varied for graphs than in the Euclidean case. The taxonomy of solutions, notations, and glossary are provided in an appendix.

4. Identification of open challenges and research directions for collaborative graph learning. The paper argues that while the three pillars (diffusion algorithms on networks, graph representation learning, collaborative learning) draw on distinct research areas, none of the existing surveys had systematically analyzed their intersection as this one does, and it identifies questions that remain unanswered. The graph portion is described as being based on a rigorous survey of recent papers using the keywords "federated graph" and "decentralized graph learning."

The authors also provide resources at https://github.com/remibourgerie/collaborative_gnns, and the paper links to a review on OpenReview.

Main Findings

• Conventional centralized ML is limited by four factors, and data access is the unconsolidated one. Deep learning's success is attributed to neural networks as universal function approximators, sample-efficient training algorithms, parallelization schemes aligned with hardware, and access to massive amounts of high-quality data. The first three have been consolidated; access to massive datasets remains a major obstacle because the data ecosystem is fragmented by privacy legislation, data-ownership concerns, integration complexity, and coordination difficulties of multi-agent systems.

• Collaborative learning offers three stated advantages over centralized learning: training and inference on distributed datasets, communication efficiency, and the promise of privacy preservation. Only model updates or aggregated information are exchanged instead of entire datasets.

• Two distinct graph scenarios emerge, versus one dominant Euclidean setting. For Euclidean data, the typical cases are cross-device (many agents, e.g., mobile phones, autonomous vehicles, mobile base stations, IoT devices) and cross-silo (a few actors with large datasets, e.g., healthcare and finance). For graph-structured data, the first scenario is learning on a large collection of independent graph instances (e.g., molecular property prediction in computational biology), where each graph instance can be treated analogously to a single data sample. The second is when agents each hold a portion of a shared global graph, so that both inference and learning require information exchange across agent boundaries (e.g., collaborative autonomous agents in IoT, autonomous vehicles, privacy-preserving learning over large-scale social network or recommender graphs).

• Message passing makes collaborative graph learning conceptually natural. Graph topology is captured by Message Passing Neural Networks (MPNNs), where nodes iteratively exchange and aggregate messages with their neighbors. Because MPNNs are inherently distributed iterative algorithms, they are conceptually well suited to collaborative settings — but when agents hold partitions of a shared graph and cannot freely share data, information must propagate through the topology, creating challenges beyond those of Euclidean settings.

• Three pillars underlie collaborative learning on graphs. Diffusion algorithms on networks (distributed iterative methods computing node- or network-level functions through local message passing, originally developed for networked systems such as multi-agent systems and sensor networks, later adopted in random-walk-based network analytics like ranking, recommendation, and semi-supervised inference); graph representation learning (deep architectures learning from graph-structured data, emerging at the intersection of graph signal processing and machine learning); and collaborative learning (e.g., federated learning, motivated by sensitive data that cannot be shared and natively distributed data that cannot be collected).

• Euclidean collaborative learning rests on assumptions that realistic systems violate. The FedAvg protocol assumes shared knowledge among agents, similar capabilities, and a common task. The survey identifies four realistic characteristics that must be addressed: data partition, statistical imbalance, system heterogeneity, and privacy vulnerabilities.

• Three partition types, beyond the standard horizontal case. Horizontal partitioning splits the sample index across agents while feature and label spaces stay the same (the default and most-studied FL setting, covering cross-device and cross-silo regimes). Vertical partitioning splits the feature and label spaces, so agents observe the same samples through distinct views (e.g., different diagnostic modalities). Transfer partitioning is a special case where source and target agents hold data from different feature spaces but share a subset of samples that act as anchor points; it often involves significant data imbalance, with the source holding far more data than the target.

• Statistical imbalance has two families. Quantity imbalance, where the number of local samples varies greatly across agents (described as a natural phenomenon: most of the world's data is concentrated among a minority of data holders), and distributional shift, split into covariate shift (label shift and feature shift, described as the most studied case for federated learning) and concept shift (where the conditional distribution varies across agents; described as rather rare and challenging for applications assuming a shared feature-to-label mapping).

• System heterogeneity covers devices, networks, and tasks. Computational heterogeneity spans IoT sensors, mobile devices, base stations, cloud containers, servers, and HPC clusters, producing order-of-magnitude differences in the time to compute local updates and in the model sizes devices can hold. Communication heterogeneity covers fiber, WiFi, 5G, and low-power IoT protocols, determining topology (star or mesh) and achievable bitrate. Slower agents, called stragglers, can become bottlenecks that delay learning or bias all participants, and dynamic populations complicate joining and leaving. Task heterogeneity arises when agents optimize distinct mappings on their data, use different architectures, or when compound models are shared.

• Federated learning does not provide inherent privacy guarantees. Despite minimal data collection and anonymization in the FedAvg design, the survey identifies leakage at multiple levels even against honest-but-curious participants: fully trained model parameters are subject to model inversion attacks that infer training labels or determine membership of specific samples; local model updates (equivalently gradients) are vulnerable to gradient inversion attacks that infer sample properties or achieve pixel-perfect data reconstruction through gradient matching, though these attacks degrade significantly with the number of agents and batch size due to aggregation; and observing the evolution of the aggregated model over training enables temporal attacks, such as inferring when samples with specific properties first appear in the training data.

• The survey deliberately scopes itself to supervised learning. References are given for collaborative unsupervised and self-supervised learning, multi-agent reinforcement learning, and reinforcement learning on graph-structured data rather than covering them directly.

Methodology in Plain English

This is a literature survey, so there is no experiment, dataset, or model training performed by the authors. Their approach has three parts.

First, they establish common ground by formalizing the collaborative learning problem for Euclidean data: K agents each sample data from an unknown local distribution, each has only a small private dataset, and the goal is to minimize the average of the agents' losses over a shared parameterized model. They describe the Federated Averaging protocol step by step — a server samples a subset of agents each round, sends them the global model, each agent runs local stochastic gradient descent steps on its private data, and the server averages the returned models — and identify the assumptions this protocol makes.

Second, they systematically list how reality departs from those assumptions, separating data partitioning (horizontal, vertical, transfer), statistical imbalance (quantity, covariate shift, concept shift), system heterogeneity (computation, communication, task), and privacy vulnerabilities (model inversion, gradient inversion, temporal observation). This gives a checklist of design concerns.

Third, they reuse that same checklist for graph-structured data. Because data can be distributed across agents in far more ways for graphs than for Euclidean data — the paper distinguishes learning on many independent graph instances from agents holding partitions of one shared graph, and notes the ways data or information can be distributed are significantly more varied — they first build a taxonomy of graph distribution scenarios, characterize the statistical heterogeneities that arise, and then follow the effectiveness/efficiency/privacy structure from the Euclidean part to see what has been solved and what remains open. An appendix holds the solution taxonomies, notation, and glossary.

Why This Matters

Impact on research. Existing surveys have covered federated learning on Euclidean data extensively and on graph data to a lesser extent, but none had systematically analyzed their intersection according to the authors. By mapping the design space and relating approaches to one another, this survey aims to let established Euclidean results accelerate work in the emerging graph setting, and to consolidate a field whose relevant knowledge is currently spread across machine learning, distributed optimization, information theory, communication systems, graph signal processing, and networked control. It also gives the graph case standardized problem formulations and a shared taxonomy, which are prerequisites for comparing methods across papers.

Real-world applications cited by the paper:

• Healthcare and finance, where large datasets are held by distinct actors who cannot share data for privacy reasons (cross-silo collaboration), including hospitals using different diagnostic modalities on shared patients.
• IoT systems, autonomous vehicles, and mobile base stations, where data is natively distributed and cannot be collected because of communication constraints.
• Computational biology, specifically predicting molecular properties, where many independent graph instances exist.
• Large-scale social networks and recommender systems, where the graph itself is split across parties that want privacy-preserving learning.

Industry relevance. The paper frames the core industrial problem as fragmented data ecosystems constrained by privacy legislation, ownership concerns, integration complexity, and multi-agent coordination. Cross-device deployments (mobile phones, IoT devices) and cross-silo deployments (banks, hospitals, e-commerce) are the two described operating regimes, and the system-heterogeneity analysis — device capability spread, network variation, stragglers, dynamic populations — speaks directly to deployment realities in wireless and edge environments. The authors also point to parallel training for large GNNs as an adjacent area with partial overlap.

Future Directions

The available excerpt states that the survey systematically identifies open challenges and promising research directions, but the truncated content does not enumerate them; the items below follow from the paper's own framing rather than reporting its final list.

• Closing the gap between Euclidean and graph methods. The authors position their contribution as extending and mapping Euclidean concepts onto graph concepts, which implies that many effectiveness, efficiency, and privacy techniques validated for Euclidean data remain to be translated, tested, or proven inapplicable for graphs.
• Handling the richer space of graph data distributions. Since the ways data or information can be distributed are significantly more varied for graphs than in the Euclidean case, the taxonomy of graph distribution scenarios and their associated statistical heterogeneities defines a set of cases whose solution methods and hardness may differ substantially from the Euclidean counterparts.
• Propagating information across agent boundaries on a shared graph. When agents hold partitions of one global graph and cannot share data, message passing must cross agent boundaries, creating novel challenges beyond Euclidean settings; designing algorithms and communication protocols for this is described as a rapidly expanding area of research.
• Understanding privacy leakage in the graph setting. The survey documents model inversion, gradient inversion, and temporal observation attacks for Euclidean collaborative learning, and notes that gradient inversion degrades with the number of agents and batch size. Whether those dynamics, and the corresponding defenses, carry over when the shared object is a graph topology rather than a feature vector is left as an open question by the framing of the privacy section.

Target Audience

Researchers and practitioners working on federated or decentralized learning who want to extend their work to graph-structured data, and graph learning researchers who want to place their methods in the collaborative learning framework. It is also useful for PhD students and engineers entering either area, because it provides unified problem formulations, a taxonomy of distribution scenarios and heterogeneities, notation, and a glossary in an appendix — and for those in applied domains such as healthcare, finance, IoT, autonomous vehicles, chemistry, and social or recommender systems who need a structured entry point into privacy-preserving learning over partitioned graphs. It is not an empirical reference: readers looking for benchmark comparisons or accuracy numbers will not find them here.

Authors’ abstract

The conventional approach to machine learning, that is, collecting data, training models, and performing inference in a single location, faces fundamental limitations, including scalability and privacy, that restrict its applicability. To address these challenges, recent research has explored collaborative learning approaches, including federated learning and decentralized learning, where individual agents perform training and inference locally, with limited collaboration. Most collaborative learning research focuses on Euclidean data with regular, grid-like structure (e.g., images, text). However, these approaches fail to capture the relational patterns in many real-world applications, best represented by graphs. Learning on graphs relies on message-passing mechanisms to propagate information between connected nodes, making it conceptually well-suited for collaborative environments where agents must exchange information. Yet, the opportunities and challenges of learning on graph-structured data in collaborative settings remain largely underexplored. This survey provides a comprehensive investigation of collaborative learning from Euclidean to graph-structured data, aiming to consolidate this emerging field. We begin by reviewing its foundational principles for Euclidean data, organizing them along three core dimensions: learning effectiveness, efficiency, and privacy preservation. We then extend the discussion to graph-structured data, introducing a taxonomy of graph distribution scenarios, characterizing associated statistical heterogeneities, and developing standardized problem formulations and algorithmic frameworks. Finally, we systematically identify open challenges and promising research directions.