From Lightweight CNNs to SpikeNets: Benchmarking Accuracy-Energy Tradeoffs with Pruned Spiking SqueezeNet

AI summary

Overview

• Research area: Energy-efficient deep learning for edge intelligence — specifically, converting compact convolutional neural networks (CNNs) into spiking neural networks (SNNs) and measuring the accuracy-versus-energy trade-off.
• Technical level: Intermediate. Readers need basic familiarity with CNNs, spiking neurons, and standard image-classification benchmarks, but the paper is a benchmarking and ablation study rather than a new theory paper.
• Scope in one sentence: The paper builds spiking versions of several lightweight CNN families (ShuffleNet, Xception, MnasNet, MixNet, SqueezeNet), evaluates them on CIFAR-10, CIFAR-100, and TinyImageNet under one training setup, and then structurally prunes SNN SqueezeNet into a variant called SNN SqueezeNet-P.

What This Paper Is About

Prior ANN-to-SNN conversion research has concentrated on large networks such as VGG and ResNet, leaving lightweight CNN-to-SNN pipelines largely unexamined. The authors fill that gap by converting compact CNN architectures into spiking networks under a single, controlled training and evaluation protocol and by recording accuracy, F1-score, parameter count, computational operations, and estimated energy consumption. They then apply structured pruning to the best-performing spiking model, SNN SqueezeNet, removing entire redundant "fire" modules to see whether a smaller spiking network can match its CNN counterpart while using far less energy.

Key Contributions

1. A systematic benchmark of lightweight SNN conversions. The authors report what they describe as the first comprehensive benchmark of state-of-the-art lightweight CNNs (ShuffleNetV2, Xception, MnasNet, MixNet, and SqueezeNet) converted into SNN counterparts and evaluated across CIFAR-10, CIFAR-100, and TinyImageNet.
2. A pruning strategy tailored to SNN SqueezeNet. From the benchmark, SqueezeNet is identified as the strongest lightweight spiking model, and the authors introduce a structured pruning method that removes whole redundant fire modules, producing SNN SqueezeNet-P.
3. A detailed CNN-versus-SNN accuracy and energy comparison. Table 2 quantifies the accuracy gap and the energy ratio for each architecture pair, including the finding that SNNs can be up to 15.7 times more energy efficient.
4. An ablation and gradient-flow analysis of pruning. Table 3 isolates which fire modules matter most, and Figure 4 shows that pruning restores gradient propagation in layers that previously suffered near-zero gradients.

Main Findings

• SNN SqueezeNet-P leads on every dataset. It achieves the best accuracy and F1-score across the board: 0.81/0.81 on CIFAR-10, 0.54/0.53 on CIFAR-100, and 0.45/0.44 on TinyImageNet.
• Pruning improves accuracy and shrinks the model. SNN SqueezeNet-P outperforms the baseline SNN SqueezeNet by 6% on CIFAR-10, 7% on CIFAR-100, and 4% on TinyImageNet, while the abstract reports a 19% reduction in parameters (CIFAR-10 parameters drop from 736.5K to 599.1K) and energy falls from 0.0355 mJ to 0.0295 mJ.
• Energy savings can be large. The abstract reports that SNNs can achieve up to 15.7 times higher energy efficiency than their CNN counterparts while retaining competitive accuracy, and that SNN SqueezeNet-P attains nearly the same accuracy as CNN SqueezeNet (only 1% lower) with an 88.1% reduction in energy consumption.
• The per-architecture energy multipliers vary widely. Table 2 reports ηE (CNN energy divided by SNN energy) values of 15.7 for MixNet, 7.0 for SqueezeNet, 5.6 for SqueezeNet-P, 5.5 for Xception, 2.5 for ShuffleNetV2, and 1.9 for MnasNet.
• ShuffleNetV2 converts poorly. Its SNN accuracy drops from 0.70 to 0.40 on CIFAR-10, which the authors attribute to channel shuffling and group convolutions interacting badly with sparse spiking activity. It is also the weakest model on CIFAR-100 (0.12) and TinyImageNet (0.11).
• Deeper is not better in the spiking domain. Xception reaches 0.75 on CIFAR-10 and only 0.23 on TinyImageNet while consuming 0.4444 mJ and 1.7797 mJ respectively, showing diminishing returns for high-capacity architectures adapted to spiking.
• MixNet is the most energy-frugal but least accurate of the mid-range models. It reports the lowest energy on all three datasets (0.0028 mJ, 0.0023 mJ, and 0.0093 mJ) with accuracies of 0.73, 0.44, and 0.34.
• MnasNet is the balanced mid-sized option. It reaches 0.78 accuracy at 0.0589 mJ on CIFAR-10, 0.49 at 0.0622 mJ on CIFAR-100, and 0.38 at 0.2486 mJ on TinyImageNet.
• The best fire-module subset is {F4, F6, F8, F9}. In the ablation, this "Ref-1" configuration achieves 0.81 accuracy with 599,130 parameters and 0.0295 mJ. Head-heavy pruning gives the lowest energy among tested schedules (0.0241 mJ at 0.77 accuracy), and the alternating schedule Alt-2 gives 0.76 accuracy at 0.0226 mJ.
• Late modules matter more than early ones, and four fire modules is the sweet spot. Tail-heavy variants beat head-heavy ones, and configurations retaining exactly four firing modules performed best.
• **Pruning repairs

Authors’ abstract

Spiking Neural Networks (SNNs) are increasingly studied as energy-efficient alternatives to Convolutional Neural Networks (CNNs), particularly for edge intelligence. However, prior work has largely emphasized large-scale models, leaving the design and evaluation of lightweight CNN-to-SNN pipelines underexplored. In this paper, we present the first systematic benchmark of lightweight SNNs obtained by converting compact CNN architectures into spiking networks, where activations are modeled with Leaky-Integrate-and-Fire (LIF) neurons and trained using surrogate gradient descent under a unified setup. We construct spiking variants of ShuffleNet, SqueezeNet, MnasNet, and MixNet, and evaluate them on CIFAR-10, CIFAR-100, and TinyImageNet, measuring accuracy, F1-score, parameter count, computational complexity, and energy consumption. Our results show that SNNs can achieve up to 15.7x higher energy efficiency than their CNN counterparts while retaining competitive accuracy. Among these, the SNN variant of SqueezeNet consistently outperforms other lightweight SNNs. To further optimize this model, we apply a structured pruning strategy that removes entire redundant modules, yielding a pruned architecture, SNN-SqueezeNet-P. This pruned model improves CIFAR-10 accuracy by 6% and reduces parameters by 19% compared to the original SNN-SqueezeNet. Crucially, it narrows the gap with CNN-SqueezeNet, achieving nearly the same accuracy (only 1% lower) but with an 88.1% reduction in energy consumption due to sparse spike-driven computations. Together, these findings establish lightweight SNNs as practical, low-power alternatives for edge deployment, highlighting a viable path toward deploying high-performance, low-power intelligence on the edge.