When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference

AI summary

Overview

Research area: Low-precision (quantized) inference in recurrent neural networks, applied to a biomedical imaging task — fluorescence lifetime imaging (FLI) for estimating molecular lifetimes from high-noise time-resolved signals.

Technical level: Intermediate. The paper assumes familiarity with recurrent networks (GRU, LSTM), quantization and quantization-aware training, but the central mechanism is explained in concrete, step-by-step terms.

Scope: The paper isolates a single operation — the rule by which a recurrent network's internal state is stored and returned to the next time step — and shows through controlled, no-retraining interventions that this rule can causally break or restore temporal inference in fixed trained models.

What This Paper Is About

Most quantization research asks how many bits a network can lose before accuracy drops. This paper asks a different question: what happens when the rule used to store a recurrent network's internal state between time steps changes, even if the network's weights stay exactly the same? Because the stored state is fed back into the next step, the storage rule becomes part of the temporal computation itself. The authors name this rule recurrent-state write-back and test it in a compact GRU encoder–decoder (Seq2SeqLite) that estimates short and long fluorescence lifetimes (τ₁ and τ₂) from noisy time-resolved signals.

Key Contributions

1. Names and isolates recurrent-state write-back as a distinct design variable. The authors formalize the mapping from the state computed at step t to the value stored and presented to step t+1, and show by changing only this mapping — with weights, gates, readout, inputs and evaluation fixed — that task performance can collapse or recover.

2. Introduces the recurrent write margin and the write deadband. The margin compares each proposed recurrent-state change against the half-step of the state quantizer (M_{t,j} = 2|δ_{t,j}| / Δ_B). Proposed changes with M < 1 fall inside the deadband, meaning deterministic nearest-level write-back keeps the previously stored value.

3. Introduces direction memory, a compact auxiliary counter that accumulates the sign of repeated sub-threshold proposed changes and advances the stored state by one quantization level once accumulated same-direction evidence crosses a threshold. Together with error feedback and quantized residual memory, it restores accuracy in frozen, un-retrained networks.

4. Distinguishes numerical fidelity from dynamical fidelity, and shows the mechanism generalizes to an independently trained 32-unit LSTM, where cell-state write-back is far more damaging than hidden-state write-back.

Main Findings

• Changing only the storage rule breaks a fixed network. On the P2F checkpoint (trained with 4-bit state write-back), continuous identity propagation yields τ₁/τ₂ lifetime RMSE of 0.36/0.35 ns, while deterministic 4-bit write-back yields 25.37/106.59 ns. The abstract reports this as roughly 70-fold and 300-fold increases for τ₁ and τ₂ respectively. Training trajectory context: P2E (continuous recurrent state) reached 1.40/3.03 ns, P2F introduced 4-bit write-back, and P3 fine-tuning recovered 0.48/0.55 ns.

• The failure is persistent suppression, not small updates per se. At the fixed P2F checkpoint, 99.59% of decoder updates lie inside the 4-bit write boundary, only 0.25% of decoder state elements change stored level between successive steps, 97.25% of decoder steps change no hidden unit at all, and an average of only 0.08 of the 32 decoder units changes level per step. Under post-training 4-bit write-back, the median completed same-direction run lasts 132 of the 134 live decoder write steps (90th percentile: 134 steps), versus a median of 2 steps for both native 4-bit and P3.

• Reconstruction error and lifetime error diverge. Deterministic P2F retains a sequence MAE of 0.09 while lifetime estimates are severely inaccurate, so a small pointwise sequence error can distort the temporal shape from which lifetime is derived.

• Independent 8-bit reference replicates the effect. Native 8-bit gives 0.20/0.22 ns; post-training deterministic 4-bit write-back raises this to 1.89/2.60 ns and pushes the decoder deadband fraction to 99.40%.

• Preserving suppressed information rescues the frozen network. On the same fixed checkpoint, error feedback recovers 0.34/0.46 ns, 4-bit residual memory 0.34/0.45 ns, and 4-bit direction memory 0.49/0.58 ns. Direction memory also shortens the median completed same-direction run from 132 to 6 steps and raises the decoder state-change fraction from 0.60% to 4.76%. At P2F, error feedback gives 0.36/0.37 ns, 2-bit residual memory 0.34/0.40 ns, and 3-bit direction memory 0.34/0.34 ns.

• Temporal memory has a boundary. The native-4-bit model is already accurate under deterministic 4-bit write-back (0.35/0.40 ns), and no memory or continuous-state alternative improves both lifetime parameters over its native interface.

• More precision can make a fixed solution worse. The 4-bit-state reference GRU degrades when its state precision is raised to 8-bit, from 0.35/0.40 ns to 0.43/0.57 ns, with continuous propagation at 0.43/0.56 ns. The finer representation is not ignored: median occupied state levels rise from 12 under 4-bit to 176.5 under 8-bit.

• Compatibility can be learned. In matched training from identical initializations across four recurrent-memory allocations (4-bit state, 6-bit state, 4-bit state with 2-bit residual memory, 4-bit state with 2-bit direction memory), direction memory gives the lowest mean τ₁ RMSE and improves on the 4-bit-state and 6-bit-state conditions in all three matched runs, while the 6-bit-state and residual-memory conditions remain stronger on τ₂.

• The LSTM reproduces the failure and shows state-specific sensitivity. The independently trained 32-unit LSTM reaches 0.239/0.254 ns under native 8-bit write-back; forcing both states to deterministic 4-bit raises this to 3.857/1.327 ns; 4-bit error feedback on both states restores 0.263/0.332 ns; continuous propagation gives 3.501/1.749 ns rather than recovering the native solution. Forcing only the cell state c to 4-bit raises RMSE to 8.158/1.292 ns, while forcing only the hidden state h gives 0.296/0.354 ns. The c-only intervention places 79.43% of decoder cell-state updates inside the write boundary with a 90th-percentile same-direction run of 121 of 134 steps, and targeted error feedback shortens that run to 4 steps and restores 0.257/0.280 ns — even though the h-only intervention has a larger deadband fraction of 93.78% with much smaller task error.

• Frozen location control. At the P2E checkpoint, encoder-only 4-bit write-back gives 2.55/1.12 ns, decoder-only gives 7.79/8.36 ns, and both regions together give 39.04/24.58 ns.

• Lifetime-conditioned behavior. Deterministic 4-bit write-back increases RMSE relative to identity propagation in all ten equal-count ground-truth bins for both τ₁ and τ₂, with the paired 95% confidence interval above zero in every bin. Excess RMSE ranges from 18.86 to 39.42 ns across the τ₁ bins and from 58.18 to 159.02 ns across the τ₂ bins, and is not monotonic with ground-truth lifetime.

Methodology in Plain English

The authors work with a fixed, already-trained model and change exactly one thing at a time. Each checkpoint is reconstructed, verified to reproduce its native implementation, and then re-evaluated with only the state-storage rule replaced — no retraining, no fine-tuning, no weight changes.

The model is Seq2SeqLite: a single-layer 32-unit GRU encoder–decoder with a linear readout and 6,627 trainable parameters, distilled from a frozen 128-unit teacher by knowledge distillation. The encoder reads 135 temporal bins of a noisy fluorescence decay and compresses it into a recurrent state; the decoder produces a temporal output sequence whose integrated shape yields the lifetime estimates.

Data: the simulated fluorescence-lifetime dataset of 1,600,000 high-noise time-resolved signals, each with 135 temporal bins, split 80/10/10 into 1,280,000 training, 160,000 validation, and 160,000 held-out test samples.

To get checkpoints at different stages of quantization, the authors built QMem, a staged quantization-aware training procedure that turns on 4-bit quantization sequentially for candidate kernels (P2A), reset-gate kernels (P2B), update-gate kernels (P2C), biases (P2D), candidate activation (P2E), and finally the recurrent state (P2F), with P3 continuing training on the fully quantized graph. During P2F the recurrence-visible state moves onto the 4-bit grid over the first 15 epochs.

The recurrent write margin makes the mechanism measurable: for each hidden unit, the proposed state change is compared with the quantizer's half-step. When the margin is below 1, the change falls inside the write deadband and the stored value does not move.

The memory interventions retain different parts of the information that cannot be written immediately: error feedback carries the discarded quantization error forward, quantized residual memory stores the discarded magnitude in a k-bit auxiliary state, and direction memory accumulates the sign of repeated sub-threshold proposals in a k-bit counter. In all cases the state returned to the network stays on the 4-bit grid. Additional experiments sweep state precision after training, compare matched training runs around different recurrent-memory interfaces, and repeat the post-training intervention in an independently trained 32-unit LSTM.

Why This Matters

Impact on research. The paper reframes low-precision recurrent inference: bit width describes how many values can be represented, but not which proposed state changes are actually written, how long discarded updates stay absent from the trajectory, or whether the resulting trajectory is compatible with the learned dynamics. A recurrent model and its state-storage rule form a coupled temporal system, so changing precision after training is a change of interface rather than a guaranteed improvement. This connects directly to the 4-bit shortfall documented in prior Seq2SeqLite compression work, where the mechanism had not been identified.

Real-world applications (all drawn from the FLI context described in the paper):

• Rapid quantitative lifetime estimation for image-guided intervention, where time-resolved fluorescence measurements must be converted to lifetime estimates with low latency.
• Tumor visualization using fluorescence lifetime contrast.
• Targeted drug delivery monitoring.
• Molecular target engagement studies and quantitative biological imaging more broadly.

Industry relevance. Quantization is a standard lever for reducing memory, arithmetic, and data-movement costs of inference, and deployment frequently maps a trained network onto a coarser representation to meet hardware constraints. This work indicates that for recurrent architectures, the state-storage interface is a first-class deployment decision: a storage rule the learned dynamics do not expect can withhold the state updates the model depends on, and cheap auxiliary memories can recover accuracy in a frozen model without retraining.

Future Directions

• Specifying write-back as a design parameter. The paper argues bit width alone is an inadequate specification; open work is defining standard metrics (deadband fraction, same-direction run length, state-change fraction) as part of deployment evaluation for recurrent models.

• Choosing memory mechanisms per architecture and state variable. The LSTM results show that cell state and hidden state have very different sensitivity, and that deadband frequency alone does not predict task damage. Which recurrent variable should receive error feedback, residual memory, or direction memory remains an open question.

• Reconciling post-training and matched-training rankings. The ordering of memory allocations changes when the interface is present during optimization rather than imposed afterward. The paper reports this as learned compatibility but does not establish a universal ranking — determining when each regime applies is unresolved.

• Extending beyond the studied architectures and task. The replication covers an independently trained GRU and LSTM on the fluorescence-lifetime inference task; whether the same failure mode and rescue transfer to other recurrent architectures, sequence tasks, or hardware-realizable write-back implementations is not established here.

Target Audience

Researchers and engineers working on quantization, low-precision inference, or deployment of recurrent models; biomedical imaging scientists using fluorescence lifetime imaging and compact sequence models for quantitative lifetime estimation; and practitioners designing hardware or software interfaces for quantized recurrent inference who need to know that the state-storage rule, not only the bit width, determines what a deployed network can compute.

Authors’ abstract

Quantization is widely used to reduce the computational and memory demands of neural-network inference. In recurrent networks, however, the quantized state is stored and returned at the next time step, so the rule used to store that state can alter subsequent computations. Here, we introduce recurrent-state write-back to denote this rule and isolate its effect in a compact GRU encoder--decoder for fluorescence lifetime imaging, a molecular imaging modality used in quantitative biological imaging. A central task is estimating two lifetime parameters, the short-lived component τ1 and the long-lived component τ2, from high-noise time-resolved fluorescence signals. Holding the trained model fixed, replacing continuous state propagation with deterministic 4-bit state storage increases estimation errors for τ1 and τ2 by approximately 70x and 300x, respectively. Failure occurs when repeated small updates remain below the write threshold, leaving the stored state nearly fixed while the network continues to propose change. Error feedback, residual memory, and direction memory carry information from these suppressed updates across time and recover accuracy without retraining. Precision sweeps show that increasing state precision can worsen a fixed recurrent solution, while matched training shows that compatibility with the state interface can be learned. To test whether this behavior extends beyond the GRU, we repeat the post-training intervention in an independently trained LSTM, where coarse write-back reproduces the failure, error feedback restores accuracy, and state-specific interventions reveal greater sensitivity of the cell state than the hidden state. Our results establish recurrent-state write-back as a key determinant of low-precision recurrent dynamics and identify the state-storage interface as a central design consideration for quantized recurrent inference.