PocketTSLM: Deploying an On-Device Soft-Prompt Time-Series Language Model for Privacy-Preserving Smartphone Reasoning

AI summary

Overview

• Research area: On-device machine learning, time-series language models (TSLMs), and mobile health / wearable sensing.
• Technical level: Advanced. The paper assumes familiarity with transformer architectures, soft prompts, LoRA adapters, tokenization, and mobile inference runtimes.
• Scope: A deployment and benchmarking study showing that the soft-prompt variant of OpenTSLM can be ported to Apple's MLX-Swift and run entirely on a consumer iPhone for EEG sleep staging and 12-lead ECG question answering, without sending health data off the device.

What This Paper Is About

Physiological time series from phones and wearables (ECG, EEG, heart rate, sleep) are usually reduced to labels, scores, or statistics, which discards the temporal evidence a reasoning system would need. Time-series language models can describe raw sensor windows in natural language, but they normally run in the cloud, and the main obstacle to running them locally is not model size but input representation: mobile LLM runtimes only accept token IDs, so a time-series encoder's embeddings cannot be injected, forcing the signal to be serialized as text and consuming the phone's limited context and memory. The paper's goal is to port the soft-prompt OpenTSLM to a smartphone so that encoded time-series windows are fed to the language model as embeddings, and then to measure what that buys in tokens, latency, and memory compared with text serialization.

Key Contributions

1. A method for migrating soft-prompt time-series language models from PyTorch to MLX-Swift, staged as PyTorch → Python MLX → Swift MLX so that framework-conversion errors can be separated from porting errors.
2. A numerical parity protocol that compares the on-device Swift implementation against both the Python MLX and PyTorch references under greedy decoding, checking token sequences rather than just final answers.
3. An on-device context-economics benchmark that compares soft-prompt encoding against a text-serialized baseline on the same Llama-3.2-1B backbone, sweeping series length and reporting time-series tokens, prompt feasibility, time to first token (TTFT), and peak memory.
4. A HealthKit-enabled iPhone demo that runs ECG and EEG reasoning locally, including on the most recent ECG read from a paired Apple Watch, with no health data transmitted during inference.

Main Findings

• Parity holds: The encoder, projector, fused encoder-projector pipeline, and rotary embedding match the Python MLX reference within a maximum element-wise tolerance of 10⁻⁴ and mean 10⁻⁵ (10⁻³ for the rotary embedding), checked on the CPU backend. The tokenizer reproduces reference token IDs exactly, and the text-serialization baseline is byte-identical to the OpenTSLM format.
• Identical decoding on device: The Python MLX and Swift implementations produce identical greedy-decoded token sequences. Against the PyTorch reference, decoded answers agree, although rationale wording can differ under bf16 arithmetic.
• On-device accuracy corroborates the port: On the first 20 samples of the Sleep-EDF CoT test split, the PyTorch reference, the Python MLX reference, and the Swift port on the iPhone all score 17/20, agree on every prediction, and make the same three errors with the same incorrect labels. The measured 85% is consistent with the 81.08% reported by Langer et al. on the full test split, though 20 samples give a 95% interval of roughly ±16 points, so this corroborates the deployment rather than estimating accuracy.
• Quality is not traded for context economy: On this task the reference reports 69.88 F1 for soft prompting versus 9.05 for the same backbone fine-tuned on a text serialization of the same signal.
• A ~12× token advantage: Soft prompts produce one token per four-sample patch (⌈L/4⌉ per channel or lead), while text serialization costs 3 BPE tokens per raw sample, giving approximately 12× fewer time-series tokens across evaluated lengths and both tasks, and 11.9× at L=250. A 12-lead ECG window of N samples costs 36N text tokens versus 12⌈N/4⌉ soft-prompt tokens.
• Context ceiling shifts substantially: Extrapolated to the backbone's 8192-token context window, text supports about 2,659 single-channel EEG samples (about 53 s of 50 Hz data) but only about 212 samples per ECG lead, whereas soft prompts support about 31,912 and about 2,553 respectively.
• Text serialization fails on 12-lead ECG: At L=100 the text arm already needs 4,133 prompt tokens and 4.3 GiB, and every longer window (L≥250, ≥9,533 tokens) was skipped to avoid an uncatchable OS memory termination. The soft-prompt arm runs the full evaluated 1,000-sample clip with TTFT ≤ 11.6 s and peak memory ≤ 4.2 GiB.
• EEG shows the same effect more gently: Text survives to L=1000 but prefills roughly 7× the tokens, so its TTFT is roughly 6× larger (14.8 ± 0.8 s versus 2.4 ± 0.1 s at L=1000) and it still exceeds the budget at the full original 1,500-sample window. Soft-prompt TTFT stays ≤ 3.0 s and peak memory ≤ 3.9 GiB at every length.
• Memory is dominated by the backbone: The bf16 Llama-3.2-1B weights alone account for about 2.3 GiB of peak memory, with the remainder growing with prompt length.
• Measurement conditions matter: All on-device numbers come from an iPhone Air running iOS 26.4.1 with MLX-Swift 0.21.3, averaged over five sweeps with greedy decoding and a maximum of 80 new tokens. EEG variance is under 6% of the mean at L≥1000 but reaches roughly 30% at L≤500; ECG variance is 19–27% at every length, driven by a faster first run. EEG thermal state rises from nominal to serious within the first sweep; ECG reads nominal at each run start even though the first run is 1.6× faster than the last, and thermal throttling was not directly established as the cause.
• Not reported: Energy consumption is not reported, and truncated EEG windows produce degenerate generated text because a window shorter than the 1,000-sample training clip is out of distribution (the reported token, latency, and memory costs are unaffected since they depend on the input encoding).

Methodology in Plain English

The team took an existing model, OpenTSLM-SP, that splits a physiological signal into short patches, encodes each patch with a small convolutional-plus-transformer encoder, and projects the results into the language model's embedding space so they sit alongside the embedded text prompt. Rather than rewriting the model from scratch, they moved it in two hops: first from PyTorch into Python MLX, then module by module into Swift MLX. Because both stages share MLX semantics, any mismatch found after the Swift port can be attributed to the port rather than to the framework conversion.

The hard part was that MLX-Swift's standard generation path only accepts token IDs, not precomputed embeddings. The authors therefore rebuilt the Llama forward pass on lower-level tensor operations so it would accept the projected time-series vectors, and reconstructed several NumPy-style convenience functions that have no direct Swift equivalent. They also had to add explicit evaluation barriers, because MLX evaluates lazily and the intermediate tensors had to be materialized at the encoder-projector hand-off, and they pinned parity tests to the CPU backend because some operations dispatch to Metal GPU kernels unavailable in headless CI.

For evaluation they used fixed, public, de-identified samples from Sleep-EDF (single-channel EEG sleep staging) and ECG-QA (12-lead ECG), with the original LoRA checkpoints and no retraining. They then ran two arms of a benchmark on identical hardware: one feeding the signal as soft-prompt embeddings, the other as the OpenTSLM text format where each z-normalized sample is rounded to two decimals, its decimal point dropped, and rendered as "012"-style text. Sweeping the series length let them see where each encoding runs out of context or memory.

Why This Matters

• Research impact: The paper moves the question for physiological time-series language models from "how good is the rationale" to "can it run where the data lives." It also isolates representation choice, not model size, as the binding constraint on mobile deployment, and provides a reusable parity protocol for porting embedding-level models to device runtimes.
• Real-world applications:
  • Local ECG interpretation on a paired Apple Watch recording, as in the demo, with no upload.
  • On-device sleep staging from EEG, where the full 1,500-sample window is out of reach for text serialization but fits in the soft-prompt budget.
  • Offline or airplane-mode clinical and field settings where network access is unavailable or untrusted.
  • Patient-facing assistants that need to ground a claim in a specific sensor window and can defer when evidence is insufficient.
• Industry relevance: Phone and wearable platforms currently treat physiological streams as summaries or text. This work shows a concrete path to keeping raw temporal evidence on the handset while still producing natural-language output, which changes the privacy posture, removes a per-query cloud cost, and gives product teams a measured budget for how much signal fits in an on-device context window.

Future Directions

• Prospective evaluation on real wearable data: The benchmarks use fixed public research samples, not prospective HealthKit records. Consumer data may include sensor noise, variable resolution, missing data, device-specific preprocessing, and population shift, so prospective studies with clinical review, IRB approval, and predefined safety boundaries are the stated next step.
• Larger backbones and quantization: Only Llama-3.2-1B was evaluated, chosen because it fits the target device. Larger models may exceed smartphone memory in this architecture, and quantization plus future hardware could raise the feasible model and input size, but neither was tested.
• Alternative fusion pathways: Only soft-prompt fusion was ported. Cross-attention fusion, as in Flamingo and OpenTSLM-Flamingo, modifies the decoder's internal attention layers rather than just the input-embedding sequence, and porting and benchmarking it on device is left to future work.
• Governance and monitoring without server logs: Local inference reduces exposure from routine cloud inference but does not remove regulatory obligations. If deployed without central telemetry, performance surveillance and drift detection would need privacy-preserving reporting, local audit mechanisms, or supervised clinical review.

Target Audience

Researchers and engineers working on on-device machine learning, mobile health systems, and time-series language models, particularly those porting embedding-level architectures to MLX-Swift or other mobile runtimes. It also suits clinical informatics and digital biomarker teams weighing local versus cloud inference, and product architects who need concrete token, latency, and memory budgets for smartphone health features. Readers looking for clinical efficacy evidence or a new modeling technique will not find it here; the contribution is deployment and measurement.

Authors’ abstract

Consumer phones and wearables increasingly collect physiological time-series data, yet most mobile systems treat these streams as summaries or text rather than as temporal evidence. This representation is poorly matched to personal health systems that need to retrieve, route, or summarize sensor evidence while preserving temporal structure. Cloud-hosted time-series language models avoid smartphone limits, but they repeatedly move sensitive physiology off the device and add latency and per-query compute cost. Mobile model alternatives that aggregate time-series data into statistical features or serialize raw samples as text either compress temporal structure or exhaust the context and memory budgets of small on-device models. We present PocketTSLM, a mobile-native deployment of OpenTSLM SoftPrompt, the soft-prompt variant of the OpenTSLM time-series language model. PocketTSLM runs fully on a consumer smartphone by injecting encoded time-series windows as embeddings instead of tokenizing time series as text. A numerical parity protocol verifies identical greedy-decoded token sequences between the Swift port and a Python reference using Apple's MLX machine-learning framework. Answers also match the PyTorch reference, although rationale wording can differ. An on-device context-economics benchmark compares the soft-prompt path with a text-serialized baseline on the same backbone. In our benchmark format, soft-prompting uses approximately 12 times fewer time-series tokens and reduces prompt-prefill cost. PocketTSLM processes 30-second Sleep-EDF electroencephalography clips for sleep staging and a 1,000-sample (10 s) 12-lead electrocardiogram clip from ECG-QA. In comparison, text-serialized baselines exceed the phone's context or memory budget at longer time-series windows. All inference runs entirely on the device, with no health data transmitted during inference.