QFabric: Guarded Cross-Domain Task Recovery and Its Limits on a Linux-RTOS Board

AI summary

Overview

Research area: Computer architecture and embedded systems — specifically heterogeneous asymmetric multiprocessing boards that pair a Linux application processor with a real-time microcontroller (MCU), and runtime task placement across that boundary.

Technical level: Advanced. The paper assumes familiarity with real-time scheduling concepts (deadlines, p95/p99 tail latency, SCHED_FIFO, admission control), inter-processor RPC transports, and empirical timing contracts.

Scope: A single-board (Arduino UNO Q) prototype and measurement study of QFabric, a guarded runtime recovery mechanism that decides whether to move a bounded task from Linux to an MCU after an observed soft real-time contract violation — and finds that on this board such a move never helps.

What This Paper Is About

Heterogeneous embedded boards combine a Linux application processor with a real-time microcontroller, which suggests an appealing idea: when a task starts missing its timing targets on Linux, move it to the MCU where it can run in isolation. This paper asks whether that move actually helps on real hardware, and how a runtime should decide when a predicted improvement is not sufficient permission to migrate. The authors build QFabric, a prototype that treats migration as a guarded hypothesis — attempted only when evidence and eligibility checks allow, and kept only after observation confirms it — and then measure its behavior against the stock Arduino Bridge RPC transport on an Arduino UNO Q.

Key Contributions

1. A guarded recovery protocol. QFabric combines a bounded wire ABI (fixed-size request/response types, versioned big-endian headers, a 988-byte payload budget), effect-based semantic admission (only pure, bounded tasks are migration eligible), quiescent epoch switching (drain in-flight work before changing domain), and probation-based commit-or-rollback of every move.

2. An interconnect characterization. The authors measure the stock Linux–MCU Bridge RPC path across payload size, concurrency, CPU load, and frequency governor, from 108,000 raw samples over 1,000 Linux-observed round trips per point.

3. A matched-load measurement study. Five workloads (add, CRC-32, and a 16-tap Q15 FIR filter repeated 1, 64, or 512 times) are profiled in both domains under idle, CPU-load, and higher-priority SCHED_FIFO contention, with ordinary and real-time callers, decomposing each call into compute time and residual dispatch cost.

4. A chain-level evaluation plus an ablation. The controller is evaluated on a replayed four-node chain against seven baselines and a non-causal offline oracle, and hysteresis and probation are isolated as necessary guards. The paper also supersedes an earlier artifact release, identifying the two measurement defects behind its previously reported 36.6% improvement.

Main Findings

• Fixed dispatch cost dominates the MCU path. Every MCU invocation pays roughly 19 ms of dispatch regardless of compute; the residual stays near 19 ms for every workload. At idle, an MCU addition takes 18.81 ms at p95 against 0.075 ms in Linux — a 251× gap.

• The MCU also computes more slowly. The MCU runs the heaviest kernel (FIR×512) 34× slower than the Cortex-A53, so heavier kernels widen the gap rather than close it.

• Payload and concurrency drive the transport. An idle zero-byte echo has p50 of 7.74 ms and p95 of 8.85 ms, sustaining 127.2 sequential calls/s; at 128 bytes, p95 rises to 34.04 ms and throughput falls to 30.2 calls/s. A 256-byte probe caused sustained loss of MCU RPC service, so the authors claim nothing beyond 128 bytes. With 1 to 8 concurrent 8-byte callers, aggregate throughput grows sublinearly from 96.75 to 257.31 calls/s (2.66×) while p95 rises from 10.27 to 29.80 ms.

• Frequency policy does not explain the loaded tail. Pinning the performance governor changed echo p95 by −11.1% to −1.3% at idle and by −0.2% to +10.6% under load, while four CPU workers raised p95 by 5.5%–23.5%.

• Linux contention hurts the MCU path more than the Linux path. Because every call is issued and relayed by Linux userspace, under RT contention the MCU path reaches p99 up to 400 ms versus 17.0 ms for Linux. Moving a task to the MCU does not move its dispatch path off Linux.

• Migration never improves a contract on this board. Across 30 cells of a four-node chain (500 observations per cell), ordinary Linux meets every contract with at most 0.8% misses, worst p95 of 26.7 ms against graph deadlines of at least 96 ms; MCU-first is several times slower in every cell. QFabric with hysteresis performs 0 switches in 30 cells — the correct decision.

• The guards matter chiefly by preventing harmful moves. Ablating hysteresis (switching on the first violating window, k=1 instead of k=3) causes switches in 7 cells under RT contention, raising p95 by up to 177× (e.g., add from 0.58 ms to 103.32 ms). Probation rolls back 5 of those moves; the remaining two were still in probation when the trace ended.

• Hardware scenarios confirm the mechanics. A controlled scenario detected a sustained violation one complete window after onset with no false switch, switched to a feasible alternate, and verified it after four windows; an injected scenario forced probation failure and produced a rollback. A negative suite exercised nine stable rejection reasons.

• The mechanism is cheap. One policy decision costs 170.1 µs of CPU time and 177.2 µs of wall time at p95 over 1,000 in-process evaluations, with a 3138-byte peak observed by tracemalloc — two orders of magnitude cheaper than one MCU dispatch. The worst LED-matrix refresh profile changes Bridge p95 by 2.05%.

• The earlier result was a measurement artifact. The superseded release reported a 36.6% p95 reduction. Its Linux executor spawned a new process per invocation (8.05 ms p95 for an addition versus 0.075 ms with a persistent executor), and its replay drew MCU latencies from idle traces while drawing Linux latencies from loaded traces, despite every MCU call being dispatched through Linux.

Methodology in Plain English

The authors start by picking a concrete platform and leaving its transport alone: the Arduino UNO Q, with a Cortex-A53 running Debian Linux and a Cortex-M33 running Zephyr, connected through Arduino's stock Bridge MessagePack RPC stack. Because the policy, the dispatcher, and the Bridge router all live in Linux userspace, every MCU call both starts and ends in Linux — a structural fact that shapes everything measured afterward.

They write each task once and generate matching implementations for both domains from that single declaration, which allows them to compare the two domains doing genuinely identical work. Before setting any timing contract, they characterize the interconnect itself: payload sweep, concurrency sweep, with and without background load, under different CPU frequency governors.

Deadlines are then fixed by rule before any replay: 1.25 times the worst idle p99 across the two domains, rounded up to the next millisecond, giving per-task deadlines of 24–58 ms. Taking the slower domain makes the contract feasible for both placements, so a violation signals degradation rather than an impossible target.

For the per-call comparison, 2,000 samples per run (after 100 warm-up) are collected in each domain under three whole-run load conditions and two caller policies, yielding 12 profile runs and 120,000 retained invocations. For the chain-level evaluation they use ordered trace replay: measured traces are composed into a four-node chain, each task visit consuming the next sample in measured order so that autocorrelation and regime changes survive. This allows paired comparisons across policies but, as the authors state, is not equivalent to running a complete application. An offline oracle that picks the faster domain per visit provides a non-causal lower bound.

Statistics are reported as p95 with 95% confidence intervals from a circular block bootstrap (2,000 resamples, block length 50). Every table, figure, and number is regenerated from raw traces by scripts in the public artifact.

Why This Matters

Impact on research. The paper reports a boundary rather than a speedup, which is unusual and useful. It shows that the isolation a real-time core offers is only available to work whose dispatch also leaves the general-purpose domain — so a whole class of "migrate to the MCU when timing degrades" proposals must first address the dispatch path, not just the compute. The explicit inequality in Section VIII-A (C_M + R_M < C_L + R_L under degradation) gives later work a concrete target to test. The verbatim correction of an earlier artifact release, with both defects named, is also a notable act of self-correction.

Real-world applications:

• Industrial control and robotics on Linux+MCU boards (motor control loops, sensor fusion) where engineers consider migrating timing-critical loops to the MCU.
• Automotive and avionics gateway ECUs using asymmetric multiprocessing with remoteproc/rpmsg or OpenAMP-style transports.
• Edge AI and vision devices where Linux handles libraries and networking while an MCU handles deterministic I/O.
• Runtime monitoring and adaptive middleware for mixed-criticality systems that need safe, reversible reconfiguration rather than one-shot placement decisions.

Industry relevance. For teams building on boards like the UNO Q, the practical takeaway is that a SCHED_FIFO caller removes most CPU-load interference and is the right first step before considering migration. The ablation shows an unguarded adaptive controller can turn sub-millisecond tails into 100-ms tails — a direct argument that guards (hysteresis, admission, probation) matter if adaptive placement is to be left enabled in production.

Future Directions

• Lower-latency transports. The paper notes shared-memory signaling reaches sub-microsecond latency on other heterogeneous SoCs; testing whether that reverses the inequality on this board is the obvious next experiment.
• Dispatch outside Linux userspace. A kernel driver or a partitioned core as the dispatcher would remove the requirement that every MCU call traverse Linux, directly attacking the dominant residual term.
• MCU-resident chains. Chains released by an MCU timer, with only inputs and results crossing the boundary, would test whether the isolation benefit can be realized at all.
• Stronger baselines and interference patterns. The authors flag SCHED_DEADLINE, CPU isolation, and affinity as comparisons not yet run, and note that synchronized or fully saturating interference would change magnitudes.
• Production hardening. State transfer, authentication, fault containment, and multi-client analysis would be required beyond the prototype's quiescent switching and effect annotations, which the authors explicitly say are not a proof of implementation equivalence.

Target Audience

Embedded systems and computer-architecture researchers working on heterogeneous Linux–RTOS platforms; engineers designing runtime adaptation, migration, or mixed-criticality middleware for asymmetric multiprocessing boards; and practitioners on Linux+MCU hardware deciding whether to move a task to the real-time core. Readers evaluating adaptive placement claims will also find the measurement-defect analysis and the ablation methodology directly useful.

Authors’ abstract

Heterogeneous embedded boards pair a Linux application processor with a real-time microcontroller (MCU), which suggests moving a task between the two domains when its timing degrades. We present QFabric, a prototype that treats cross-domain placement as guarded recovery from a violated empirical soft real-time contract: a bounded task is compiled for both domains, monitored in windows with hysteresis, switched only after semantic and capacity admission at a quiescent epoch boundary, and committed or rolled back after probation. On an Arduino UNO Q (Cortex-A53 Linux and a Cortex-M33 running Zephyr), we characterize the stock Bridge RPC interconnect and measure five workloads in both domains under idle, CPU-load, and higher-priority SCHED_FIFO contention, with ordinary and real-time callers. Each MCU invocation costs about 19 ms of dispatch regardless of compute, the MCU runs the heaviest kernel 34x slower, and because every call traverses Linux userspace, Linux contention inflates the MCU path far more than the Linux path (p99 up to 400 ms versus 17.0 ms). Across 30 cells of a four-node chain, Linux meets every contract within 0.8% misses and QFabric correctly never migrates; a controller without hysteresis migrates in 7 cells, raising p95 by up to 177x, and probation rolls back 5 of those moves. On Linux-dispatched heterogeneous boards, migration therefore cannot isolate a task from Linux interference, and recovery guards matter chiefly by preventing harmful moves. This paper supersedes an earlier artifact release whose reported improvement arose from measurement defects that we identify. All source, raw traces, and generators are public.