SNAP: Low-Latency Test-Time Adaptation with Sparse Updates

AI summary

Overview

Research area: Test-Time Adaptation (TTA) for deep neural networks under distribution shift, with a focus on on-device / edge deployment.

Technical level: Intermediate. The paper assumes familiarity with batch normalization, pseudo-labeling with confidence scores, and the standard TTA family (Tent, EATA, SAR, CoTTA, RoTTA), but the core ideas are explained in accessible terms.

Scope: The paper proposes SNAP, a framework that makes existing test-time adaptation methods run with far fewer model updates on resource-constrained edge devices, by selecting a small set of representative samples and correcting normalization statistics at inference time.

What This Paper Is About

Test-time adaptation lets a model adjust itself to new data distributions using only unlabeled test samples, but state-of-the-art TTA methods do heavy computation (backpropagation, data augmentation, model ensembling) on every batch, which is far too slow for devices like a Raspberry Pi. Simply adapting less often saves time but sharply degrades accuracy, because only a small portion of the data stream is then used for adaptation. SNAP's goal is to make infrequent, sparse adaptation work well by choosing which samples to adapt on, and by correcting the model's normalization statistics between adaptation steps.

Key Contributions

1. A Sparse TTA (STTA) framework: SNAP introduces an Adaptation Rate (AR) that controls the proportion of batches or samples used for adaptation relative to original TTA, allowing a tunable trade-off between latency and accuracy. SNAP maintains competitive accuracy even when adapting on only 1% of the incoming stream.

2. Class and Domain Representative Memory (CnDRM): A memory that stores a small pool of samples that are simultaneously class-representative (high pseudo-label confidence, prediction-balanced across classes) and domain-representative (closest to the target domain centroid in early-layer feature statistics, measured by Wasserstein distance). Memory size is fixed to match the batch size.

3. Inference-only Batch-aware Memory Normalization (IoBMN): A normalization scheme that runs at inference with no backpropagation. It takes the memory samples' normalization statistics and corrects them toward the current inference batch using a soft shrinkage function, keeping the model aligned with the evolving data distribution between sparse updates.

4. Integration with five SOTA TTA algorithms (Tent, EATA, SAR, CoTTA, RoTTA) across CIFAR10-C, CIFAR100-C, and ImageNet-C, with latency measured on three real edge devices. SNAP reduces latency by up to 93.12% while keeping accuracy drop below 3.3%, across adaptation rates from 1% to 50%.

Main Findings

• Latency reduction on real hardware: On a Raspberry Pi 4 testbed, SNAP reduced CoTTA's latency by up to 93.12% at an adaptation rate of 0.1 on CIFAR10-C, with no loss in performance. Baseline existing TTA methods incurred a minimum latency of 3.83 seconds per batch on Raspberry Pi 4, and CoTTA required over 70 seconds per adaptation step.

• Sparse adaptation alone is not enough: Naïve STTA with an adaptation rate of 0.33 (adapting once every three batches) reduced latency proportionally to the rate but caused a significant accuracy drop compared with fully adapting original TTA (Figure 1).

• Accuracy comparable to full adaptation across rates: SNAP maintained performance comparable to original TTA methods across adaptation rates from 0.01 to 0.5, achieving 77.12%–81.74% for Tent, close to the full adaptation accuracy of 80.43%. At higher adaptation rates (0.5 or 0.3), SNAP can even surpass fully adapted methods by filtering out harmful data points.

• Biggest gains for the most expensive methods: CoTTA benefits most from SNAP because it updates all model parameters and is therefore highly dependent on an effective sampling strategy. In Table 1 (AR = 0.1, CIFAR100-C, Raspberry Pi 4), CoTTA dropped from 49.39% accuracy to 35.86% under naïve STTA, while CoTTA + SNAP reached 50.52%.

• CIFAR100-C results at AR = 0.1 (Table 1): Tent went from 55.76% (full) to 52.84% (STTA) to 55.84% with SNAP; EATA from 50.74% to 49.70% to 52.35%; SAR from 57.94% to 53.49% to 55.76%; RoTTA from 51.55% to 49.11% to 51.33%.

• ImageNet-C results at AR = 0.1 (Table 1): Tent went from 39.53% to 35.47% (STTA) to 39.03% with SNAP; SAR from 39.34% to 38.62% to 41.99%; CoTTA from 28.72% to 27.81% to 30.19%.

• Domain-representative sampling matters (preliminary experiment): On ImageNet-C Gaussian noise, TTA with the closest 20% of samples to the domain centroid achieved 26.65% accuracy, whereas the farthest 20% showed 18.52%.

• Ablation on CIFAR10-C at AR = 0.1 (Table 2, averaged over all 15 corruptions): CnDRM + IoBMN produced the best results for every backbone tested — Tent 78.95 ± 0.09, CoTTA 78.83 ± 0.06, EATA 78.61 ± 0.13, SAR 78.06 ± 0.07, RoTTA 77.07 ± 0.10. The CnDRM-only setting raised CoTTA from 66.42 ± 0.12 (Naïve) to 77.69 ± 0.10, showing that sampling was the dominant factor for that method.

• Low-entropy sampling underperforms: Contrary to the belief that low-entropy samples benefit TTA, the LowEntropy setting performed worse than Random for STTA (the provided text is truncated mid-sentence at this point).

• Memory efficiency: SNAP operates efficiently under memory constraints with low memory overhead and integrates seamlessly with a memory-efficient TTA module (MECTA, Hong et al., 2023), as detailed in Appendices B.7 and B.8. Whether SNAP changes memory consumption by a specific measured amount is not reported in the provided text.

Methodology in Plain English

The researchers start from the observation that on edge devices, the adaptation step — not the inference step — is the bottleneck. So they keep the model's normal inference path intact and only occasionally let it learn from the test stream.

Choosing what to learn from. Instead of adapting on every batch, SNAP keeps a small memory whose capacity equals the batch size. As each unlabeled sample arrives, the model makes a prediction. Samples whose predicted class probability exceeds a confidence threshold τ_conf are candidates for memory. But rather than filling memory with whatever is most confident, SNAP balances selections across predicted classes so that no class dominates. Separately, for each candidate the method looks at the mean and variance of the features entering the first normalization layer — early layers tend to retain domain-specific information. It computes the Wasserstein distance between a sample's feature statistics and a running "domain centroid" (updated by momentum with β = 0.9) and prefers samples close to that centroid, since those best represent the current domain. When the memory is full, the sample farthest from the centroid within an overrepresented class is evicted.

Adapting rarely. The model updates only every k batches, where the adaptation rate is 1/k. Between updates, the model is frozen.

Staying aligned between updates. Because the memory's statistics can go stale between updates, SNAP adds IoBMN, which runs only at inference. For each channel it compares the current batch's normalization statistics with the memory's statistics. If the difference is small, it leaves the memory statistics alone; if large, it nudges them toward the batch using a soft shrinkage function, so that small deviations are ignored and large ones are only partially corrected. A parameter α (set to 4 in the experiments) controls how much to trust the memory versus the current batch. No gradients are computed, so this step adds negligible cost.

Testing. The authors vary the adaptation rate over 0.01, 0.03, 0.05, 0.1, 0.3, and 0.5; report mean accuracy and standard deviation over three random seeds; and measure latency on Raspberry Pi 4, Raspberry Pi Zero 2 W, and NVIDIA Jetson Nano. They use CIFAR10-C, CIFAR100-C, and ImageNet-C (15 corruption types, five severity levels, using the highest severity), with ResNet18 pretrained on CIFAR10/CIFAR100, and ResNet50 and ViT-Base pretrained on ImageNet from TorchVision. Generalization is also checked on ImageNet-R and ImageNet-Sketch (Appendix B.11).

Why This Matters

Impact on research. Most TTA research optimizes accuracy on servers and treats latency as an afterthought. This paper reframes latency as the primary constraint and shows that the standard evaluation protocol — adapt on every batch — hides a large practical failure mode. It also challenges a widely held assumption in TTA, showing that low-entropy sample selection is worse than random selection in the sparse regime, which suggests that existing sample-filtering heuristics (EATA, SAR) do not transfer directly to low adaptation rates. SNAP is also designed as a drop-in module rather than a replacement algorithm, so it composes with future TTA methods.

Real-world applications (from the paper):

• Autonomous driving, where delay in processing sensor data can lead to dangerous situations.
• Real-time health monitoring on wearable or portable devices.
• Mobile devices and IoT sensors with limited computational power.
• High-frame-rate video or high-performance sensor streams, where the model must keep pace with the data or miss critical samples.

Industry relevance. The target deployment class — Raspberry Pi 4, Raspberry Pi Zero 2 W, NVIDIA Jetson Nano — is exactly the hardware used in edge AI products. A framework that cuts per-batch latency by up to 93.12% with accuracy loss below 3.3% and low memory overhead makes previously server-only TTA methods plausible on shipping devices. Source code is available at https://github.com/chahh9808/SNAP under a CC BY-NC-ND 4.0 license.

Future Directions

• Joint tuning of adaptation rate and inference correction: The α parameter in IoBMN (fixed at 4 in these experiments) and the momentum β (0.9) were chosen empirically. Whether these can be adapted automatically per stream speed or per device is not addressed in the provided text.
• Extension beyond image classification: All reported benchmarks are image corruption and domain-shift datasets (CIFAR10-C, CIFAR100-C, ImageNet-C, ImageNet-R, ImageNet-Sketch). Applicability to streaming video, time-series sensor data, or language models is not reported.
• Understanding when low-entropy selection fails: The paper reports that low-entropy sampling hurts sparse TTA, contrary to prior belief, but the provided text is truncated before the full explanation. A principled theory of sample selection under very low adaptation rates remains open.
• Interaction with memory-efficient TTA: The paper reports low memory overhead and compatibility with MECTA, but the detailed memory measurements (Appendices B.7 and B.8) are not included in the provided text, so the precise memory-versus-latency trade-off at each adaptation rate is not reported here.

Target Audience

Researchers and engineers working on test-time adaptation, domain adaptation, or on-device / edge machine learning will get the most from this paper. It is also relevant to practitioners building latency-sensitive deployed systems (autonomous driving, wearables, industrial IoT) who need to know whether TTA is viable on their hardware, and to TTA method designers who want their algorithms to remain effective when updates are sparse. Readers should be comfortable with batch normalization and the standard TTA literature; the paper's own components are presented with enough detail to reimplement.

Authors’ abstract

Test-Time Adaptation (TTA) adjusts models using unlabeled test data to handle dynamic distribution shifts. However, existing methods rely on frequent adaptation and high computational cost, making them unsuitable for resource-constrained edge environments. To address this, we propose SNAP, a sparse TTA framework that reduces adaptation frequency and data usage while preserving accuracy. SNAP maintains competitive accuracy even when adapting based on only 1% of the incoming data stream, demonstrating its robustness under infrequent updates. Our method introduces two key components: (i) Class and Domain Representative Memory (CnDRM), which identifies and stores a small set of samples that are representative of both class and domain characteristics to support efficient adaptation with limited data; and (ii) Inference-only Batch-aware Memory Normalization (IoBMN), which dynamically adjusts normalization statistics at inference time by leveraging these representative samples, enabling efficient alignment to shifting target domains. Integrated with five state-of-the-art TTA algorithms, SNAP reduces latency by up to 93.12%, while keeping the accuracy drop below 3.3%, even across adaptation rates ranging from 1% to 50%. This demonstrates its strong potential for practical use on edge devices serving latency-sensitive applications. The source code is available at https://github.com/chahh9808/SNAP.