Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federation

AI summary

Overview

Research area: Federated and decentralized machine learning, specifically privacy-preserving collaborative fine-tuning of large language models.

Technical level: Intermediate. Readers should be comfortable with federated learning concepts, parameter-efficient fine-tuning (LoRA), knowledge distillation, and multi-armed bandit basics.

Scope: This paper proposes KNEXA-FL, a hybrid architecture that keeps model training fully decentralized while adding a lightweight, non-aggregating central matchmaker that learns which peers should exchange knowledge, and demonstrates its advantages over random pairing and centralized distillation on a heterogeneous code-generation task.

What This Paper Is About

Organizations want to fine-tune LLMs on their own private data without sharing it, but federated learning's usual server-based design creates a single point of failure and a juicy target for model-inversion attacks. Decentralized alternatives remove the server but typically pair peers at random, which wastes opportunities and can actively harm participants when mismatched models are forced together. KNEXA-FL's goal is to keep the privacy and robustness benefits of decentralized training while making the collaboration graph itself intelligent.

Key Contributions

1. A hybrid orchestrated-decentralized architecture. KNEXA-FL introduces a Central Profiler/Matchmaker (CPM) that never aggregates model parameters and never accesses agent models or raw data. It only observes abstract profile vectors and issues pairing directives, preserving the security posture of decentralized FL while restoring intelligent coordination.

2. P2P LLM collaboration framed as a contextual bandit. The authors are the first to cast peer selection in a federated LLM setting as a contextual combinatorial bandit problem, solved with LinUCB over privacy-preserving agent profiles so that the collaboration graph improves online as rewards accumulate.

3. Adaptive Knowledge Distillation (AKD) for heterogeneous models. Rather than exchanging parameters or logits, paired agents exchange decoded text predictions on a shared transfer set. The receiving model re-tokenizes the teacher's text with its own tokenizer, which makes distillation well-defined even when backbones, tokenizers, and architectures differ.

4. Empirical validation against strong centralized baselines. On a deliberately heterogeneous six-client code-generation federation, KNEXA-FL achieves roughly 50% relative Pass@1 improvement over random P2P and converges stably, whereas centralized distillation collapses catastrophically.

Main Findings

• Learned matchmaking beats random and heuristic pairing. KNEXA-FL reaches 13.33% Pass@1 versus 8.89% for Random-P2P (about a 50% relative gain) and 6.67% for a heuristic that greedily maximizes data divergence. The heuristic underperforming random pairing shows that naive diversity maximization can be actively harmful.

• Centralized distillation is unstable under heterogeneity. The Central-KD baseline peaked at 18.33% Pass@1 and then collapsed to 2.00%, while the newer FedID-CentralKD variant failed to converge at all (1.11%). The authors attribute this to a single averaged "ensemble teacher" overwriting specialized client knowledge, causing catastrophic forgetting.

• The quality of knowledge transfer is dramatically better under CPM guidance. On the held-out 128-problem transfer set, the best student under CPM pairing hit 86.70% Pass@1, roughly 2.6x the best result achieved by any random pairing (33.33%).

• The CPM learns a compatibility trade-off, not just maximum diversity. In a controlled synthetic ablation, LinUCB pairing peaked at a 48.5% relative gain with 32 clients in a high-heterogeneity setting, and gains shrink to ~3.1% in low-heterogeneity settings. The learned policy accepts slightly lower Jensen-Shannon divergence (~0.64) than a naive diversity-maximizing baseline (0.710) in exchange for better synergistic compatibility.

• Gains are distributed across the federation, including weaker models. Client C2 (bloom-560m) rose to the strongest individual model with 36.67% local Pass@1, and even the smallest model (pythia-410m) substantially exceeded its isolated performance.

• Overhead is manageable. Each LinUCB update costs O(d_p²), and approximate nearest-neighbor pre-filtering reduces matchmaking from O(N²d_p) to O(N·k·d_p). A 20-agent round completes in under 16 minutes on eight A100 GPUs.

Methodology in Plain English

The system has three parts. A set of autonomous agents each hold a frozen base LLM plus a small trainable LoRA adapter, fine-tuned locally on private, non-IID data. A central matchmaker holds no models and no data — it only receives compact profile vectors describing each agent's model family, configuration, task performance, and past collaboration outcomes. Finally, matched pairs communicate directly over an encrypted channel.

Each round proceeds in four phases. First, agents train locally and emit updated profiles. Second, the matchmaker forms a context vector for every candidate pair by combining their profiles with the current network state, scores each pair with an upper confidence bound that balances exploiting known-good matches against exploring untested ones, and greedily selects a set of disjoint pairs. Third, matched agents exchange knowledge: the teacher runs its model on a shared, privacy-vetted transfer set of prompts and sends back the decoded text, which the student re-tokenizes and distills into its own adapter via cross-entropy loss. Fourth, the receiving agent computes a scalar reward — the reduction in its local loss minus a penalty for communication bytes — and reports it back, which updates the matchmaker's linear model.

The evaluation uses a merged HumanEval/MBPP code-generation benchmark, split unevenly across clients with a Dirichlet distribution to simulate realistic non-IID conditions, and evaluated with Pass@k and CodeBLEU against a suite of baselines: isolated training, two centralized distillation schemes, a diversity-maximizing heuristic, and random pairing.

Why This Matters

Impact on research. The paper reframes a problem the federated learning community has largely treated as a systems or optimization question — who should collaborate with whom — as an online learning problem. It shows that a lightweight, non-aggregating coordinator can capture much of the benefit of central orchestration without inheriting its security liabilities, and it provides direct evidence that centralized aggregation of heterogeneous LLMs can fail catastrophically rather than merely underperform.

Real-world applications:

• Cross-hospital medical NLP. Hospitals that cannot share patient records could collaboratively tune clinical language models while a neutral coordinator (a regulator, consortium, or standards body) merely decides which institutions should exchange knowledge.

• Financial services consortia. Banks subject to strict data-sovereignty rules could improve fraud-detection or compliance models by pairing with peers whose data distributions are complementary, without any party holding another's model.

• Multi-vendor edge and telecom deployments. Firms deploying heterogeneous, vendor-supplied models across regional sites could improve each site's model without a central model store, which matters where latency, regulation, or vendor IP blocks centralization.

• Cross-jurisdictional enterprise AI. Multinationals legally barred from moving data across borders could still let regional models learn from one another through distilled text predictions on a shared, non-sensitive prompt set.

Industry relevance. The paper is authored by Fujitsu Research of Europe and accompanied by a public implementation, signaling a systems-and-deployment orientation rather than pure theory. The low communication cost of text-based distillation, the modest matchmaker overhead, and the alignment with data-space governance concepts (e.g., International Data Spaces) make the approach plausible for regulated enterprise settings where a trusted but non-privileged broker already exists.

Future Directions

• Scaling and realistic networking. The evaluation covers 6 real clients and synthetic federations up to 64. Validating on larger fleets with realistic WAN latencies, stragglers, and asynchronous participation remains open.

• Richer matchmaking models. The CPM uses a linear LinUCB model over hand-designed profile features. Neural or otherwise more expressive bandit models could capture non-linear compatibility structure, though at a cost to interpretability and update efficiency.

• Stronger formal privacy and verifiability. The authors outline differential privacy on logits, zero-knowledge proofs for profile attestations, and token-efficient disparity audits (referencing TFDP) as additions to the current guardrail-filter and encrypted-channel design.

• More realistic heterogeneity and data splits. Beyond Dirichlet partitioning, future work should test semantically structured splits (e.g., user-profile-based), and benchmark against a wider range of advanced centralized FL optimizers, particularly ones designed to resist catastrophic forgetting.

Target Audience

This paper is most valuable to federated learning and distributed ML researchers, particularly those working on peer-to-peer and serverless training architectures, knowledge distillation across heterogeneous models, and privacy-preserving LLM adaptation. It is also directly relevant to applied ML engineers and architects in regulated industries (healthcare, finance, telecom) evaluating whether collaboration across organizational boundaries is feasible without a trusted parameter server, and to researchers interested in bandit-based resource allocation and multi-agent coordination under privacy constraints. Readers seeking a formal convergence proof will not find one here — the theoretical section is explicitly framed as intuition linking LinUCB regret bounds to the spectral properties of the induced collaboration graph.

Authors’ abstract

Fine-tuning Large Language Models (LLMs) for specialized domains is constrained by a fundamental challenge: the need for diverse, cross-organizational data conflicts with the principles of data privacy and sovereignty. While Federated Learning (FL) provides a framework for collaboration without raw data exchange, its classic centralized form introduces a single point of failure and remains vulnerable to model inversion attacks. Decentralized FL (DFL) mitigates this risk by removing the central aggregator but typically relies on inefficient, random peer-to-peer (P2P) pairings, forming a collaboration graph that is blind to agent heterogeneity and risks negative transfer. This paper introduces KNEXA-FL, a novel framework for orchestrated decentralization that resolves this trade-off. KNEXA-FL employs a non-aggregating Central Profiler/Matchmaker (CPM) that formulates P2P collaboration as a contextual bandit problem, using a LinUCB algorithm on abstract agent profiles to learn an optimal matchmaking policy. It orchestrates direct knowledge exchange between heterogeneous, PEFT-based LLM agents via secure distillation, without ever accessing the models themselves. Our comprehensive experiments on a challenging code generation task show that KNEXA-FL yields substantial gains, improving Pass@1 by approx. 50% relative to random P2P collaboration. Critically, our orchestrated approach demonstrates stable convergence, in stark contrast to a powerful centralized distillation baseline which suffers from catastrophic performance collapse. Our work establishes adaptive, learning-based orchestration as a foundational principle for building robust and effective decentralized AI ecosystems.