Scalable Explainability-as-a-Service (XaaS) for Edge AI Systems

AI summary

Overview

Research area: Explainable AI (XAI) systems for edge and IoT deployments, combining distributed systems design, caching architectures, and model interpretability.

Technical level: Intermediate. The paper assumes familiarity with XAI methods (LIME, SHAP, GradCAM), edge computing, and service-oriented architectures, but its architectural arguments and evaluation are accessible without deep mathematics.

Scope: The paper proposes and empirically evaluates XaaS, a distributed service architecture that decouples explanation generation from model inference so that edge devices can request, cache, verify, and reuse explanations across heterogeneous hardware.

What This Paper Is About

Most explainable AI methods run "coupled" to inference, meaning every explanation is computed on the spot, even for inputs that closely resemble earlier ones. On resource-constrained edge devices this creates redundant computation, high latency, and poor scaling across heterogeneous hardware. The authors propose Explainability-as-a-Service (XaaS), which treats explanations as a first-class system service with semantic caching, lightweight verification, and adaptive method selection, rather than as a model-specific feature.

Key Contributions

1. A decoupled XaaS architecture comprising five components: an edge device layer, a two-tier distributed explanation cache, an explanation generation layer, a verification module, and a service orchestrator. Inference and explanation generation are separated so each can be optimized independently.

2. A distributed explanation cache with semantic similarity retrieval. The system embeds inputs (CLIP for images, BERT or Sentence-BERT for text) and uses FAISS for nearest-neighbor search over cached explanations, with four formalized validity conditions: semantic similarity, prediction consistency, model version, and fidelity preservation.

3. A lightweight verification protocol that uses n = 15 perturbations, versus over 1000 for full LIME and thousands for full SHAP, to check whether cached explanations remain valid after model updates or input drift.

4. An adaptive explanation engine that selects both an explanation method (LIME, SHAP, GradCAM, Integrated Gradients, attention mechanisms) and a location (device, edge, or cloud) by minimizing computation plus communication time, evaluated across three real-world edge-AI use cases.

Main Findings

• Latency reduction: XaaS achieved a mean end-to-end latency of 42 ms (95% CI: [39 ms, 45 ms]), a 38% reduction versus EdgeXAI (68 ms), 56% versus CloudXAI (95 ms), and 67% versus LocalGen (127 ms). The authors attribute this mainly to a 72% cache hit ratio from adaptive method selection.

• Explanation fidelity: Mean fidelity was 0.923 (95% CI: [0.915, 0.931]), meeting the 0.92 target. CloudXAI and LocalGen reached 0.945 and 0.941 on their initial explanations, but only XaaS verifies cached explanations after model updates or input drift.

• Throughput and load handling: XaaS sustained 187 req/s (95% CI: [175 req/s, 199 req/s]), compared with 98 req/s (EdgeXAI), 72 req/s (CloudXAI), and 58 req/s (LocalGen), a 3.2x increase over LocalGen. Under peak load of 250 req/s, XaaS reached a 98.5% success rate while all other systems fell below 90%.

• Per-scenario performance (Table II): Manufacturing quality control (MQC) showed 38 ± 3 ms latency, 0.928 fidelity, 74.2% cache hit rate, 195 req/s, and 42% latency improvement over the best baseline. Autonomous vehicle fleet (AVF) showed 48 ± 4 ms, 0.915 fidelity, 68.5% hit rate, 172 req/s, and 31% improvement. Healthcare monitoring (HCM) showed 40 ± 3 ms, 0.926 fidelity, 73.8% hit rate, 194 req/s, and 40% improvement.

• Cache dynamics: Hit ratios rose rapidly in the first 3-4 hours and stabilized at 74% (manufacturing), 69% (automotive), and 74% (health). Manufacturing benefited most from repetitive patterns. Returns diminished beyond 2000 entries: a 1000-entry cache achieved 72% hit rate at 42 ms, while doubling to 2000 entries yielded only 75% hits at 40 ms. The authors conclude the 10,000-entry global cache is optimal.

• Verification effectiveness: With model updates every 6 hours, the protocol detected 95.5% of invalidated explanations using only 3.2% of full regeneration computation. False positives occurred at 2.8%, causing minor delay without affecting correctness. Without verification, the paper reports that only 27.7% of cached explanations are invalid.

• Scalability: Increasing device count 100x raised XaaS latency by only 37% (38 ms to 52 ms), versus EdgeXAI (77%), CloudXAI (54%), and LocalGen (42%). At a 300 req/s peak, success rates were 97.1% (XaaS), 79.3% (EdgeXAI), 72.5% (CloudXAI), and 61.8% (LocalGen).

• Ablation results: Removing caching increased latency by 86% (fidelity stayed at 0.918). Removing verification lowered latency to 51 ms but reduced fidelity to 0.887 (−3.9%), below the 0.90 threshold, due to stale explanations. Removing adaptive selection raised latency to 64 ms (+52%). All components together gave the best combination: 42 ms and 0.923 fidelity.

• Robustness: With model retraining every six hours, XaaS maintained fidelity above 0.91. Under ±30% network latency jitter, XaaS latency increased 16% to 49% but remained below all baselines. Shifting the device capability mix (from 20-80% to 80-20% low-tier devices) changed performance by no more than ±8%. Adding Integrated Gradients and attention methods increased selection time by under 5 ms and improved fidelity by 2.1%.

• Statistical significance: Improvements were significant in all five trials (paired t-test, p < 0.001), with effect sizes of 2.87 versus LocalGen, 2.14 versus CloudXAI, and 1.93 versus edge generation without caching.

Methodology in Plain English

The authors first formalize the "edge explainability problem": N devices, each with a model, compute capacity, bandwidth, and latency tolerance, served by M more powerful edge servers. Each request carries a fidelity threshold and a latency bound, and the goal is to minimize total cost (computation plus communication) subject to those constraints. The naive approach, generating every explanation independently, wastes work because similar inputs with the same prediction tend to have similar explanations.

XaaS replaces that naive approach with three mechanisms. First, instead of storing explanations by exact input, the system stores them by embedding vector and retrieves near matches using FAISS nearest-neighbor search, accepting a cached explanation only if it satisfies four conditions: the embeddings are close enough, the predictions match, the model version is current (or passes verification), and the cached fidelity meets the request's threshold.

Second, because models are periodically retrained, cached explanations can go stale. The verification protocol tests a cached explanation against 15 fresh perturbations rather than the 1000+ used by full LIME or SHAP, which the authors position as a cheap staleness check.

Third, on a cache miss, a greedy search picks the cheapest method-and-location combination that meets the fidelity requirement, with a complexity of O(|M| · |L|).

The authors evaluate on three scenarios: manufacturing quality control (150 devices, 127K sampling points, 8 defect classes), autonomous vehicle fleet (80 vehicles, 215K driving scenarios), and healthcare monitoring (200 patients, 89K vital sign samples). Hardware included Raspberry Pi 4B, NVIDIA Jetson Nano, and Intel NUC devices, five edge servers per scenario (Tesla T4 GPU, 1000-entry cache), and a cloud server with a 10K global cache. Baselines were LocalGen (on-device), CloudXAI (cloud SHAP/LIME), EdgeXAI (edge generation without caching), and FedXAI (federated XAI). Settings were ε_sim ∈ [0.12, 0.18], φ_target = 0.92, and n = 15 verification samples, with a 2-hour warm-up, 24-hour run, and five repetitions per experiment.

Why This Matters

Impact on research: The paper reframes explainability from a per-model algorithmic property into a system-level service with cache consistency, verification, and delivery semantics, drawing an explicit analogy to content delivery networks. It argues that prior XAI work (LIME, SHAP, GradCAM, saliency approximations, distilled explainer networks) never addressed creation, storage, or reuse of explanations at the system level, and that prior ML-as-a-service and transparency-as-a-service proposals lacked concrete architectural designs.

Real-world applications:

• Manufacturing quality control with repetitive defect patterns, where the highest cache hit rate (74.2%) was observed.
• Autonomous vehicle fleet perception, where more diverse inputs lowered the hit rate to 68.5%.
• Healthcare monitoring, where explanations must meet fidelity thresholds for clinical accountability.
• Large-scale heterogeneous IoT deployments where devices range from Raspberry Pi 4B to Jetson Nano and Intel NUC.

Industry relevance: As regulatory pressure to justify automated decisions grows, the paper positions XaaS as a way to embed audit and quality control into heterogeneous edge deployments. It claims gradual integration into existing ML serving systems through APIs, and notes that reusing one LIME explanation (127 ms) can serve 8 to 12 additional explanation requests, amortizing operational cost.

Future Directions

1. Federated XaaS: Enabling collaborative distributed learning while keeping data local, which would require explaining global models from local data and managing cache consistency across federations.

2. Counterfactual explanations: Extending caching beyond feature-attribution methods to counterfactual explanations, which are more input-specific and would require new semantic similarity measures.

3. Continual learning integration: Using incremental verification or partial cache invalidation to reduce overhead as models are continuously updated.

4. Security and privacy: Integrating differential privacy or secure multi-party computation, and defending against cache poisoning and explanation inference attacks while balancing privacy with explanation utility and system performance.

Target Audience

This paper is most useful for edge AI and IoT system architects, MLOps and ML serving engineers, and XAI researchers interested in deployment rather than algorithm design. It also suits practitioners in regulated domains such as manufacturing, automotive, and healthcare who need auditable explanations under latency and resource constraints. Readers seeking formal optimization guarantees or new explanation algorithms will find the work light on theory; the authors themselves present practical approximations rather than exact solutions to the optimization problem.

Authors’ abstract

Though Explainable AI (XAI) has made significant advancements, its inclusion in edge and IoT systems is typically ad-hoc and inefficient. Most current methods are "coupled" in such a way that they generate explanations simultaneously with model inferences. As a result, these approaches incur redundant computation, high latency and poor scalability when deployed across heterogeneous sets of edge devices. In this work we propose Explainability-as-a-Service (XaaS), a distributed architecture for treating explainability as a first-class system service (as opposed to a model-specific feature). The key innovation in our proposed XaaS architecture is that it decouples inference from explanation generation allowing edge devices to request, cache and verify explanations subject to resource and latency constraints. To achieve this, we introduce three main innovations: (1) A distributed explanation cache with a semantic similarity based explanation retrieval method which significantly reduces redundant computation; (2) A lightweight verification protocol that ensures the fidelity of both cached and newly generated explanations; and (3) An adaptive explanation engine that chooses explanation methods based upon device capability and user requirement. We evaluated the performance of XaaS on three real-world edgeAI use cases: (i) manufacturing quality control; (ii) autonomous vehicle perception; and (iii) healthcare diagnostics. Experimental results show that XaaS reduces latency by 38% while maintaining high explanation quality across three real-world deployments. Overall, this work enables the deployment of transparent and accountable AI across large scale, heterogeneous IoT systems, and bridges the gap between XAI research and edge-practicality.