LAFA: Agentic LLM-Driven Federated Analytics over Decentralized Data Sources

AI summary

Overview

• Research area: Privacy-preserving data analytics, specifically the intersection of LLM-agent-based data analytics and federated analytics (FA) over decentralized data sources.
• Technical level: Intermediate. Readers need basic familiarity with LLM prompting and agent architectures (ReAct, Chain-of-Thought), plus a working understanding of federated analytics and privacy techniques such as differential privacy, secure multi-party computation, and homomorphic encryption.
• Scope: The paper proposes and evaluates LAFA, a hierarchical multi-agent system that compiles natural language analytical queries into optimized, executable federated analytics workflows, and reports completion-ratio and operation-count results on 20 generated queries over two query sets.

What This Paper Is About

LLM agents can turn plain-language questions into multi-step analytics plans, but today's frameworks assume the data is centralized, which conflicts with privacy regulations and creates exposure risks. Federated analytics keeps raw data on user devices and shares only privacy-protected intermediate results, but it requires structured, machine-readable queries and has little support for natural language. LAFA is presented as the first system to combine the two: it accepts natural language queries on the front end and emits valid, optimized FA execution plans on the back end.

Key Contributions

1. First LLM-driven federated analytics framework. LAFA is claimed as the first system to integrate LLM-agent-based data analytics with FA, supporting complex natural language queries while preserving privacy.
2. Hierarchical multi-agent design. A coarse-grained planner splits a complex query into single-intent sub-queries, and a fine-grained planner maps each sub-query into a preliminary Directed Acyclic Graph (DAG) of FA operations using stored FA DAG templates, reducing sequencing errors.
3. DAG optimizer agent. An optimizer agent rewrites and merges the preliminary DAGs into one optimized posterior DAG by merging overlapping operations, simplifying redundant or overly broad operations, adding implied operations, and preserving valid FA execution order.
4. Evaluation on complex real-world-style queries. Experiments across two query sets (AdultPii and Apple) measure query decomposition quality, privacy compliance, and resource efficiency against zero-shot and one-shot single-agent prompting baselines.

Main Findings

• Completion ratio: LAFA reaches a near-perfect completion ratio of 95 to 100 percent (100 percent on AdultPii, 95 percent on Apple), compared with 10 to 15 percent for zero-shot prompting (15 percent AdultPii, 10 percent Apple) and 60 to 75 percent for one-shot prompting (75 percent AdultPii, 60 percent Apple), across 20 queries per dataset.
• Failure modes of baselines: Zero-shot prompting often omits critical operations in the final calculation phase (for example, it may perform filtering, encryption, aggregation, noise addition, and decryption but omit the final calculation needed for averages or ratios). One-shot prompting captures most required operations but struggles to identify relationships and distinctions among sub-queries on complex queries.
• Lower resource-intensive operation counts: Against one-shot prompting, LAFA achieves an average reduction of 1.35 to 1.40 in access (Acce), encryption (Enc), and aggregation (Aggr) operations, and 1.15 to 1.25 in DP noise-addition and decryption (Dec) operations.
• AdultPii operation counts: One-shot prompting averages Acce 3.55, Enc 3.55, Aggr 3.55, DP 4.95, Dec 4.95, Cal 3.85; LAFA averages Acce 2.20, Enc 2.20, Aggr 2.20, DP 3.70, Dec 3.70, Cal 5.50.
• Apple operation counts: One-shot prompting averages Acce 3.40, Enc 3.40, Aggr 3.40, DP 4.60, Dec 4.60, Cal 3.45; LAFA averages Acce 2.00, Enc 2.00, Aggr 2.00, DP 3.45, Dec 3.45, Cal 5.25.
• Work is shifted to lightweight post-processing: LAFA reuses shared intermediate results across sub-queries instead of repeating secure computation, and offloads complexity to cheaper final calculations such as division, subtraction, and ratio computation.
• Ablation — structural priors are essential: Removing the preliminary DAG knowledge drops the completion ratio to 0 percent, with misaligned edge dependencies especially between data filtering and encryption and between decryption and subsequent computation.
• Ablation — hierarchy matters: Replacing the hierarchical planner with a single flat planner reduces the completion ratio to 35 percent; the paper attributes partial recovery to the robustness of the DAG optimizer.
• Ablation — optimizer efficiency gains: Removing the DAG optimizer raises per-client access, encryption, and aggregation operations from 2.20 to 3.75, and DP plus decryption cost from 3.70 to 5.65, while calculation operations drop from 5.50 to 3.65. The completion ratio remains 100 percent with or without the DAG optimizer.
• Not reported: The paper does not report wall-clock runtime, communication volume in bytes, or measured privacy-loss (epsilon) values for the evaluated queries.

Methodology in Plain English

LAFA is described as an LLM-powered compiler for federated analytics, sitting on top of an existing FA engine rather than introducing new cryptographic protocols. Three entities are involved: queriers who submit natural language questions, client devices that each hold exactly one data record, and a server that hosts the aggregator plus the multi-agent system.

On the server, a coarse-grained planner agent breaks a complex question into sub-queries, each with a single analytical intent, using cues such as "and" or "vs." A fine-grained planner agent then matches each sub-query to a stored FA DAG template, drawn from the canonical pipeline of preprocessing, encryption, aggregation, noise addition, decryption, and postprocessing, and instantiates it with the relevant data features. A DAG optimizer agent then merges the preliminary DAGs into one optimized graph, reusing shared filtering, encryption, and aggregation steps and inserting any implied operations such as differences or ratios. Finally, an answerer agent turns the aggregated results back into natural language.

Evaluation uses 20 natural language analytical queries generated with GPT-4o: one set based on the schema of a commonly used FA dataset (AdultPii, 32,563 records and 18 features per individual) and one set synthesized from Apple's differential privacy report using a retrieval-augmented generation approach. Two metrics are used: completion ratio (proportion of queries producing a complete, valid FA DAG) and operation count (access, encryption, aggregation, DP noise addition, decryption, and calculation operations). Baselines are zero-shot and one-shot prompting of a single agent. A lightweight structure checker validates DAG nodes and edges, with manual inspection of intermediate and final computation results. All experiments use GPT-4 via the OpenAI API with temperature set to 0. The threat model assumes honest-but-curious clients contributing only local records, a semi-honest server that sees only encrypted or noise-protected aggregates, and a differentially private released result; privacy guarantees are inherited from backends such as Arboretum and Orchard.

Why This Matters

Impact on research. The paper positions itself at the seam between two otherwise disconnected lines of work: LLM-agent analytics systems that assume unrestricted data access (ReAct, InsightPilot, InfiAgent-DABench, AutoTQA) and FA systems that support only structured, task-specific queries (Honeycrisp, Orchard, Arboretum). It argues that the orchestration layer, not the cryptographic layer, is the missing piece, and it provides an evaluation protocol (completion ratio plus per-operation counts) for that layer.

Real-world applications:

• University or enterprise workforce analytics, such as comparing average salary across role groups without collecting individual salary records (the paper's running example).
• Consumer device telemetry, such as emoji usage, Safari diagnostics, QuickType suggestions, and health data tracking analytics, mirroring the Apple differential privacy use cases used to generate evaluation queries.
• Healthcare-style subgroup analysis where demographic or health attributes must remain on user devices.
• Any regulated deployment under GDPR or CCPA that needs to answer ad hoc subgroup-comparison questions without centralizing personal data.

Industry relevance. Because LAFA explicitly does not propose new FA protocols and builds on standardized backends, the design is meant to be adopted incrementally by organizations that already run an FA stack and want a natural language interface. The reported reductions in encryption, aggregation, and decryption counts target the cost that makes federated deployments expensive in practice.

Future Directions

• Support for heterogeneous schemas and client populations. LAFA assumes a horizontally partitioned dataset where each client device holds exactly one record with a consistent, fixed feature schema; relaxing this assumption is not addressed by the paper.
• Validation beyond generated queries. All 20 queries per dataset were generated by GPT-4o from a dataset schema and from Apple's differential privacy report using retrieval-augmented generation because the authors report no real-world complex natural language query dataset. Collecting genuine user queries would test the approach under realistic phrasing.
• Measured end-to-end resource costs. The paper reports operation counts rather than wall-clock latency, transfer volume in bytes, or server/client load figures for its own runs; measuring these would substantiate the efficiency claims directly.
• Deeper privacy accounting. The paper inherits differential privacy guarantees from the backend and does not report privacy-loss values; quantifying the end-to-end privacy budget consumed by LLM-generated plans remains open. It also notes that LAFA stays compatible with future improvements in FA primitives, implying the orchestration layer should be retested as those evolve.

Target Audience

Researchers and practitioners working on privacy-preserving machine learning, federated learning and analytics, and LLM agent systems. It is also relevant to engineers building natural language interfaces over regulated or distributed data, and to applied scientists evaluating whether agentic planning can be trusted to produce protocol-valid execution graphs. Readers without grounding in FA operation ordering will find the system-design and evaluation sections accessible but should expect to consult the background section for the privacy primitives.

Authors’ abstract

Large Language Models (LLMs) have shown great promise in automating data analytics tasks by interpreting natural language queries and generating multi-operation execution plans. However, existing LLM-agent-based analytics frameworks operate under the assumption of centralized data access, offering little to no privacy protection. In contrast, federated analytics (FA) enables privacy-preserving computation across distributed data sources, but lacks support for natural language input and requires structured, machine-readable queries. In this work, we present LAFA, the first system that integrates LLM-agent-based data analytics with FA. LAFA introduces a hierarchical multi-agent architecture that accepts natural language queries and transforms them into optimized, executable FA workflows. A coarse-grained planner first decomposes complex queries into sub-queries, while a fine-grained planner maps each subquery into a Directed Acyclic Graph of FA operations using prior structural knowledge. To improve execution efficiency, an optimizer agent rewrites and merges multiple DAGs, eliminating redundant operations and minimizing computational and communicational overhead. Our experiments demonstrate that LAFA consistently outperforms baseline prompting strategies by achieving higher execution plan success rates and reducing resource-intensive FA operations by a substantial margin. This work establishes a practical foundation for privacy-preserving, LLM-driven analytics that supports natural language input in the FA setting.