The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks

AI summary

Overview

Research area: Neuromorphic computing / spiking neural networks (SNNs) — specifically energy-efficient inference through sparsity, applied to image classification (CIFAR10, CIFAR100, ImageNet) and event-based neuromorphic classification (DVS-CIFAR10, DVS-Gesture).

Technical level: Intermediate to Advanced. Readers should be comfortable with leaky integrate-and-fire (LIF) neuron dynamics, backpropagation through time (BPTT), and surrogate gradients.

Scope: The paper proposes the Dormant Spiking Neuron (DSN), a state-driven gating mechanism that permanently silences neurons whose membrane potential falls below a negative threshold, and evaluates the resulting energy savings and accuracy across five benchmarks and two SNN architectures.

What This Paper Is About

SNNs save energy mainly because they skip work when inputs are zero (input-driven sparsity). But a receiving neuron that never fires still performs membrane-potential integration and leakage at every timestep. The authors measure this and find that 44% of neurons in an ImageNet-trained ResNet-18 SNN sit in a deeply inhibited state (V_m ≤ 0) with a negligible chance of firing, yet keep consuming computation. The goal is to let such neurons switch off entirely — per input sample, not permanently — and to train that switching behavior end-to-end so accuracy is not sacrificed.

Key Contributions

1. The DSN model. A state-driven sparsity paradigm in which each neuron carries a binary dormancy gate that irreversibly closes once its membrane potential drops below a negative threshold, bypassing all subsequent computation for that neuron within the sample.
2. Orthogonality to input-driven sparsity. Because the gate acts on postsynaptic (output-column) activity while conventional SNN sparsity acts on presynaptic (input-row) activity, the two compound multiplicatively in the synaptic operation (SOP) count.
3. An end-to-end training formulation. A full BPTT derivation with surrogate gradients applied to both the spike-firing nonlinearity and the dormancy gate, so the network learns a dormancy policy rather than applying a fixed post-hoc rule.
4. Broad empirical validation. Results on CIFAR10, CIFAR100, ImageNet, DVS-Gesture and DVS-CIFAR10 with MS-ResNet and Spike-Driven Transformer backbones, including ablations on threshold choice, timestep count, surrogate-gradient vs. straight-through estimator, and instance-level mask diversity.

Main Findings

• The overlooked overhead is quantified: in an ImageNet-trained ResNet-18 SNN, roughly 26% of neurons are in the active spiking region, 30% in the subthreshold active region (0 < V_m < V_th), and 44% in the deeply inhibited region (V_m ≤ 0).
• Energy savings of 40% to over 70% are reported across static and neuromorphic datasets while maintaining accuracy comparable to or better than baseline SNNs. Energy figures are analytical estimates from a hardware model, not device measurements.
• ImageNet headline result: DSN (BPTT) with ResNet-18 reaches 64.44% accuracy at 1.17 mJ estimated energy, versus the MS-ResNet baseline's 63.10% at 4.29 mJ — an estimated 73% energy reduction and a 1.34% accuracy improvement.
• Beats listed static pruning methods on ImageNet: at comparable reported cost reductions, the compared static methods (e.g., AT-LIF 59.61% at ~77% reduction, DPAP 60.41% at ~51.7%, Unstructured 61.89% at ~77%, UniSparse 61.89% at ~78%, SEENN ResNet-34 67.48% at ~55.3%) fall below the 63.10% baseline, whereas DSN improves on it.
• Transformers respond well: on CIFAR100, DSN (BPTT) with the 2-512 Spike-Driven Transformer reaches 79.36% at 0.33 mJ versus the baseline's 78.40% at 0.58 mJ — an accuracy increase of nearly 1%.
• Neuromorphic data: on DVS-Gesture, DSN (BPTT) matches the baseline's 96.88% while consuming 42% less estimated energy; on DVS-CIFAR10 it saves over 52% energy with a 0.2% accuracy drop (77.9% vs. 78.1%). Extending to T = 24 on DVS-CIFAR10 gives a 0.4 percentage-point deviation from baseline.
• Savings grow with simulation length: on CIFAR100 with ResNet-18, energy savings are approximately 30% at T = 2, 45% at T = 4, over 63% at T = 16, and exceed 73% at T = 96.
• Two deployment modes behave differently: inference-time optimization (ITO) is training-free but incurs a modest accuracy penalty (e.g., 62.04% on ImageNet ResNet-18 vs. 63.10% baseline); BPTT training preserves or improves accuracy but requires full retraining.
• Savings come from dormancy, not from suppressing firing: the neuronal computational utilization (NCU) metric, defined as R̄ + (1 − D̄), exceeds 70% across most layers of the ImageNet model, indicating the gains come from increasing the dormancy fraction among redundant neurons rather than lowering the firing rate.
• Dormancy masks are input-specific: across 10,000 ImageNet image pairs on ResNet-34, average Jaccard overlap of dormant masks is 0.40, with more than 12 layers below 0.20 — versus a fixed similarity of 1 for static pruning.
• Surrogate gradient beats straight-through: the smooth surrogate for the dormancy gate improves accuracy by 0.30% on CIFAR10 and 0.63% on CIFAR100, and reaches 64.44% on ImageNet versus the STE baseline, a 1.57% margin.
• Training cost is modest but real: for ResNet-18 on ImageNet with batch size 128 on a single GPU, baseline SNN achieves 742.4 images/s and 15.3 GB memory; DSN-BPTT achieves 601.6 images/s and 20.5 GB — a 19% throughput reduction and 34% memory increase.
• Run-to-run stability: three-seed means are 95.30 ± 0.10 (CIFAR10), 76.61 ± 0.13 (CIFAR100), 64.36 ± 0.19 (ImageNet), 96.65 ± 0.40 (DVS-Gesture), 77.83 ± 0.12 (DVS-CIFAR10).
• Threshold choice: D_th = −0.05 is used for static datasets and D_th = −0.5 for neuromorphic datasets; the CIFAR100 sweep shows energy sits on a low plateau for |D_th| ≤ 0.1 and rises sharply for more conservative thresholds.

Methodology in Plain English

The authors modify a standard LIF neuron with a four-step loop per timestep. First, a one-bit "dormancy gate" checks whether the neuron's membrane potential at the previous timestep fell below a negative threshold; if so, the gate closes and stays closed for the rest of that sample. Second, only active neurons integrate their decayed potential with weighted incoming spikes. Third, a spike is emitted only if the neuron is active and its integrated potential crosses the firing threshold. Fourth, the membrane potential is updated, with a hard reset for neurons that fired. Because a dormant neuron can never fire again, its potential updates need never be executed.

Training uses backpropagation through time. Both the spike-firing step function and the dormancy-gate step function are non-differentiable, so the forward pass uses the exact step function while the backward pass substitutes a smooth surrogate derivative. The authors derive the full set of backward recurrences over the spike, potential, and gate-state error terms, then compute weight gradients by summing contributions across timesteps.

For efficiency accounting, they express the SOP count as MACs multiplied by the sum over timesteps of the mean input spike rate times the mean fraction of non-dormant neurons — the product form being what makes the two sparsity types multiplicative rather than merely additive.

Empirically, they take two spike-driven architectures (MS-ResNet and the Spike-Driven Transformer) implemented in SpikingJelly, and evaluate them under two paradigms: ITO (apply the dormancy rule to an already-trained network) and BPTT (train from scratch with the gate in the loop). Ablations sweep the dormancy threshold over three seeds, vary the simulation length T, compare surrogate gradient against a straight-through estimator, and use SSIM plus a spike-train-agreement metric to check whether ITO preserves the original network's internal representations.

Why This Matters

Impact on research. The paper reframes SNN efficiency: the dominant narrative is presynaptic event sparsity, and this work argues that postsynaptic subthreshold state maintenance is a distinct and large cost that existing pruning methods — which remove structures based on dataset-level statistics — do not address. Framing the two sparsity types as orthogonal and multiplicative gives a clean accounting model that other efficiency papers can adopt. The Jaccard analysis (0.40 average overlap vs. a fixed 1 for static pruning) provides direct evidence that the redundancy being exploited is instance-level rather than dataset-level.

Real-world applications:

• Always-on edge vision sensors and battery-constrained cameras where every millijoule of inference energy matters.
• Event-camera gesture and motion recognition on wearable or embedded neuromorphic processors.
• Neuromorphic co-processors in robotics, where latency and power budgets constrain continuous perception.
• Deployment of larger SNN backbones on fixed-power budgets, since savings grow with simulation length (over 73% at T = 96).

Industry relevance. The mechanism is cheap to implement — a 1-bit mask per neuron that lets the hardware bypass 32-bit weight-memory accesses and potential updates — and the authors point to a neuromorphic design (Zhou et al., 2026) reporting over 98% of invalid computations bypassed in self-attention workloads and synaptic energy efficiency below 0.06 pJ/SOP. The 19% training-throughput and 34% memory penalty is a one-time cost against a persistent inference saving, which is the favorable side of the trade for deployment-heavy use cases.

Future Directions

1. Optimizing the subthreshold active region. The NCU analysis shows a residual gap attributable to neurons that neither fire nor go dormant; the authors explicitly flag fine-grained optimization of this population as future work.
2. Extending beyond rate and multi-spike coding. The authors state the mechanism depends on informational redundancy and may be less suitable for low-redundancy schemes such as Time-to-First-Spike coding, where a single spike's timing carries the information.
3. Hardware realization and measurement. All reported energy figures are analytical estimates; actual device-level energy and latency remain to be measured on silicon implementing the 1-bit dormancy mask.
4. Combining with early-exit and dynamic-timestep strategies. The finding that the DSN already saves roughly 30% at T = 2 suggests compatibility with low-latency regimes, but this combination is not tested.

Target Audience

Graduate students and researchers in neuromorphic computing and energy-efficient deep learning; hardware architects designing sparsity-aware SNN accelerators; and practitioners deploying SNNs on edge or event-driven platforms who want a training-free (ITO) or retrained (BPTT) route to lower inference cost. Readers unfamiliar with LIF dynamics and surrogate-gradient BPTT will find the method section demanding, though the motivation and results tables are accessible with a general deep-learning background.

Authors’ abstract

Spiking neural networks (SNNs) offer a compelling pathway toward energy-efficient artificial intelligence, fundamentally anchored in presynaptic input-driven sparsity. However, a substantial yet chronically overlooked source of computational overhead persists: the continuous subthreshold dynamics of postsynaptic neurons. Quantitative analysis reveals that up to 44% of neurons in an ImageNet-trained ResNet-18 model reside in a deeply inhibited state, imposing a persistent computational burden despite having a negligible probability of firing. To address this inefficiency, the Dormant Spiking Neuron (DSN) model is introduced. By incorporating a state-driven dynamic gate, the DSN allows a neuron to enter a dormant state when its membrane potential falls below a predefined negative threshold, dynamically pruning redundant computations at the instance level. Crucially, this dormancy mechanism is directly integrated into the neuronal dynamics and trained end-to-end via backpropagation through time using surrogate gradients, enabling the network to actively learn a robust dormancy policy that adaptively preserves critical information flow. Extensive experiments across static and neuromorphic datasets demonstrate that the DSN framework consistently reduces computational energy by 40% to over 70%. Notably, on the large-scale ImageNet dataset, the DSN outperforms state-of-the-art static pruning techniques at comparable reported cost reductions and simultaneously improves the corresponding baseline SNN accuracy by 1.34%. By establishing a trainable, state-driven sparsity paradigm orthogonal to conventional input-driven methods, the DSN unlocks synergistic efficiency gains, presenting an effective approach for advancing neuromorphic computing.