Energy and Latency First: How Deployment Constraints Reverse Time-Series Model Selection

AI summary

Overview

Research area: Machine learning systems and efficiency — specifically time-series model selection under deployment constraints, applied to cellular handover prediction at the network edge.

Technical level: Intermediate. The paper is readable without deep telecom background, but familiarity with model benchmarking, latency/energy measurement, and basic classification metrics (F1, AUC) helps.

Scope: An evaluation of eight time-series architectures for handover prediction on user equipment, ranked by a protocol that treats the 40 ms inference deadline as a hard feasibility filter and measured inference energy as the selection cost, rather than accuracy alone.

What This Paper Is About

Machine learning models for time series are usually chosen by accuracy, then deployed on devices where they must finish within a hard deadline and run within a cumulative energy budget — constraints that never appeared in the ranking. The authors use handover prediction in cellular networks as a concrete, measurable test case: the model runs on the user equipment once every 40 ms measurement period and its prediction is worthless if it arrives late. The goal is to show what happens to model rankings when deadline and energy are made explicit parts of selection.

Key Contributions

1. An evaluation protocol that screens candidate models for deadline feasibility at the tail (p99 latency) rather than the mean, then ranks the survivors on measured inference energy — counting feature computation that precedes the forward pass and is usually omitted from reported costs.
2. A benchmark of eight architectures under that protocol, compared against what an accuracy-only benchmark would select, covering gradient-boosted trees, convolutional, recurrent, attention, random-feature and canonical-feature models.
3. Ablations that separate feature representation from model class (LightGBM and an MLP on 247 engineered features versus LightGBM and CNN-1D on the raw flattened window), plus a seven-width capacity sweep and a knowledge-distillation experiment testing whether compression can rescue the neural route.
4. Three stated conditions under which the observed outcome should generalize beyond telecommunications: a simple or known decision boundary, a short input sequence, and duty-cycled inference.

Main Findings

• The tree model dominates the entire frontier: LightGBM is the most accurate model (F1 = 0.974, AUC 0.998), the lowest-latency one (0.381 ms p99), and the cheapest to run under a duty-cycled deployment pattern (4,947 μJ per inference), 2.2× below the next cheapest of the eight.
• The accuracy ranking is not the deployment ranking: CNN-1D is the best neural model on accuracy (F1 = 0.914) but costs over 3× the per-inference energy of LightGBM under duty-cycled measurement, and is less accurate as well. Every neural model is dominated.
• Cheap representations alone do not suffice: MiniRocket reaches F1 = 0.872 but its 10k-kernel transform pushes full-pipeline latency to 1.32 ms, comparable to the deep models; catch22+ridge collapses to F1 = 0.371, indicating that the specific structure LightGBM's features capture is what carries the task.
• All models clear the deadline comfortably: even p99 latency is at least 20× below the 40 ms bound, so every model is feasible and selection reduces to energy. The slowest (catch22+ridge, p99 = 1.98 ms) would stay feasible on a chip roughly 20× slower; LightGBM (p99 = 0.38 ms) tolerates 105×.
• The measurement pattern itself changes the conclusion: busy-loop measurement understates deployment energy for every model but by very different factors — 0.30 for LightGBM versus 0.95–0.98 for CNN-1D and Efficient-HO-v2. Measured as a tight loop, LightGBM appears 10.4× cheaper than CNN-1D; measured duty-cycled, the advantage narrows to 3.3×.
• Omitting feature computation nearly doubles the tree model's apparent cost: for LightGBM the preprocess stage (0.116 ms) is 44% of total inference and nearly equals the forward pass; MiniRocket's ridge head is negligible but its transform dominates.
• The tree's advantage comes from model class, not privileged features: LightGBM gains only 0.003 from engineered features over the raw window (0.974 vs 0.971), and a two-layer MLP on the same 247 features (0.929) beats CNN-1D on raw input (0.919).
• Compression cannot close the gap: the capacity sweep rises monotonically from F1 = 0.814 at 1,921 parameters to 0.924 at 58,033 parameters, still short of 0.974; distilling CNN-1D into the 7,618-parameter Efficient-HO yields no improvement (student F1 falls from 0.438 to 0.407, mean gain −0.031 ± 0.033), and an apparent +0.08 gain in earlier 50 s experiments was a data-volume effect.
• Prediction can still help inside the existing rule: although no neural model is selected, an LSTM used as a confidence gate for early handover execution saves an average of 542 ms of Time-to-Trigger latency per handover at 100% success rate, reducing ping-pongs from 5 to 2.

Methodology in Plain English

The authors build a simulated but realistic cellular environment: a 3D model of Santa Clara, California, ray-traced with Sionna for radio signal strength and driven by SUMO vehicular traffic, giving 7 base station sites, 21 cells, 100 user devices over 100 seconds at a 40 ms sampling cadence — 4.7M measurements containing 596 handover events.

The prediction task is to look at the last 8 timesteps of signal strength across the 21 visible cells and decide whether the standardized 3GPP A3 handover condition will fire within the next 8 timesteps. Because that condition is a written-down rule (a neighbour exceeding the serving cell by a hysteresis of 1 dB plus an offset of 2 dB, sustained for 8 steps), the labelling function is known in closed form — the models are asked to recover a rule that already exists rather than discover an unknown one. Labels are computed on the full continuous traces and then windowed, with negatives subsampled to roughly 30% positive; splits are 70/15/15 and stratified, and all results average five seeds.

Eight models are trained: three neural baselines representative of the handover literature (LSTM, Transformer, CNN-1D), three feature-based models (LightGBM on 247 engineered features, MiniRocket, catch22+ridge), and two deliberately small "capacity probes" (Efficient-HO and Efficient-HO-v2) that locate the low end of the neural capacity range.

Costs are measured on an isolated 36-core Xeon CPU — chosen because the device runs inference on its application processor, not an accelerator. Each inference is split into preprocess, forward and postprocess, and all reported latency and energy cover the whole pipeline. Energy is measured twice: in a busy loop with the core continuously active (what throughput benchmarks do), and duty-cycled in bursts of 50 inferences separated by 10 ms idle pauses (what a deployed device actually does). Power is inferred from CPU utilization at 50 ms resolution, so absolute microjoule values are estimates while relative comparisons on one platform hold.

Why This Matters

Impact on research: The paper argues that accuracy-only benchmarking does not merely under-report deployment cost — it can rank first a model that cannot be deployed. It also shows that the measurement protocol is itself a result: the same models, measured a different way, produce a different margin (10.4× versus 3.3×), so published efficiency claims are protocol-dependent. The three generalizability conditions give other subfields a checklist for when to expect the same inversion.

Real-world applications:

• Handover prediction and proactive mobility management on user equipment in 5G and beyond, where cells are smaller and handovers more frequent and directional.
• Other always-on sensor inference on battery-powered devices, where a model runs once per sampling period for the life of the deployment.
• Industrial and automotive edge prediction, where a late output is superseded by the event it was predicting.
• Any deployment where FLOPs or parameter count are used as a proxy for device energy, since the paper shows these proxies miss the idle-dominated cost structure.

Industry relevance: The work comes from Ericsson Research and targets a function being standardized rather than merely proposed. Its practical message is that a gradient-boosted tree on cheap features can beat deep sequence models on accuracy, latency and energy simultaneously — and that training cost is a non-issue: LightGBM trains 40.6× faster and uses 37.7× less energy than CNN-1D while reaching the highest validation accuracy, amortized within seconds at 25 inferences per second.

Future Directions

• Test beyond the urban simulation: the dataset is one ray-traced Santa Clara scene; generalization to other environments, mobility patterns and field deployments is untested, and the simulations are limited in temporal diversity.
• Replace utilization-based energy estimates with hardware counters: the authors note microjoule figures are estimates derived from CPU utilization at 50 ms resolution, and absolute values will differ on handset silicon.
• Establish when the inversion holds elsewhere: the three conditions (known decision boundary, short sequence, duty-cycled inference) are stated as predictions; validating them on longer sequences, unknown labelling functions, or server-side batched inference would test the boundary of the claim.
• Explore hybrid use of neural models despite their cost: the LSTM early-execution experiment, which saved 542 ms of Time-to-Trigger per handover, suggests a path where prediction accelerates an existing rule rather than replacing it — a direction the paper reports but does not optimize.

Target Audience

Practitioners and researchers selecting models for edge or embedded deployment — especially in telecommunications, IoT and embedded ML — who need to justify architecture choices on latency and energy rather than accuracy alone. It is also useful for benchmarking methodologists interested in how measurement protocol shapes reported results, and for engineers who rely on FLOPs or parameter counts as efficiency proxies and want evidence of where that breaks down.

Authors’ abstract

AI models, and time-series models in particular are typically ranked by accuracy and deployed under constraints absent from that ranking. In this paper, we study a real setting where these constraints are explicit and measurable: handover pre- diction in cellular networks, where inference runs on the User Equipment once per 40 ms measurement period and must complete before the network acts. The timing bound is hard: a late prediction is superseded by the measurement it was predicting, while energy is a cumulative budget spent for the life of the device. Neither appears in the objective these models are selected on. We evaluate eight architectures under a protocol that screens for deadline feasibility at the tail rather than the mean, then ranks the feasible models on measured inference energy, count- ing the feature computation that precedes the forward pass and is usually omitted. Gradient-boosted trees on handcrafted temporal features are the most accurate model (F1=0.974) and among the cheapest to run, while the neural architecture an accuracy-only benchmark would rank first (CNN-1D, F1=0.914) costs over 3 times more energy per inference under a realistic duty-cycled deployment pattern. We give three conditions under which this outcome should be expected, so the finding applies beyond telecommunications. Accuracy-only benchmarking does not merely under-report deployment cost. It can rank first a model that cannot be deployed.