LEVIO: Lightweight Embedded Visual Inertial Odometry for Resource-Constrained Devices

AI summary

Overview

Research area: Embedded computer vision and robotics — specifically visual-inertial odometry (VIO) designed for ultra-low-power hardware.

Technical level: Advanced. The paper assumes familiarity with feature-based visual odometry, bundle adjustment, nonlinear optimization, and embedded system design concepts such as memory hierarchies and multicore parallelization.

One-sentence scope: The paper presents LEVIO, a six-degrees-of-freedom monocular VIO pipeline built in C for microcontrollers and low-power systems-on-chip, validated on the GAP9 RISC-V SoC at 20 FPS while consuming less than 100 mW, and benchmarked on the EuRoC dataset.

What This Paper Is About

State-of-the-art VIO systems achieve high tracking accuracy but require computing platforms that draw watts to tens of watts, which excludes micro-drones, nano-drones, and smart glasses with strict payload and power limits. Existing low-power alternatives get around this by simplifying the problem — for example, reducing tracking to 2D planar optical flow — or by building custom ASICs, which are not deployable on commercial off-the-shelf hardware.

LEVIO's goal is to close that gap: keep the full-featured components of accurate VIO pipelines (ORB features, bundle adjustment, tightly coupled inertial fusion) while restructuring the algorithm and its implementation so it fits and runs in real time on a commercially available ultra-low-power multicore SoC.

Key Contributions

1. An efficient six-DoF VIO pipeline. Through design-space exploration using a Python golden model, the authors built LEVIO from established components — ORB feature tracking, the 8-point algorithm, EPnP RANSAC, keyframe selection, triangulation, IMU pre-integration, and bundle adjustment — into a slim design able to process image streams at 20 FPS.

2. Hardware-software co-design for embedded targets. The pipeline was tailored to microcontrollers and SoCs with strict memory limits. This includes a custom linear algebra library (supporting SVD of square matrices, the Jacobi Eigenvalue algorithm, the Inverse Power method, and linear system solvers), a hierarchical memory handler moving data between L1 and L2, reduced QQVGA (160 x 120) image resolution, a Schur-complement formulation of the optimizer, and core-level parallelization. The reported memory footprint is below 256 kB for VO without optimization and below 1 MB for the whole pipeline.

3. On-hardware validation. The full pipeline was implemented for the GAP9 SoC by GreenWaves Technologies and benchmarked against the public EuRoC VIO dataset, achieving 20 FPS at less than 100 mW.

4. Open-source release. The complete implementation is released at https://github.com/ETH-PBL/levio.

Main Findings

• Accuracy of the full pipeline: Full LEVIO reaches the highest accuracy of the tested configurations when adequately parametrized, with an average absolute RMSE of 3.46 m over all five EuRoC sequences.

• IMU data mainly reduces long-term drift: The relative mean translation error over 40 m sub-trajectories is in a similar range with and without IMU data, but the full-trajectory RMSE is lower for the full pipeline. On the MH05 sequence, the relative error for both pipelines is nearly identical, while the absolute RMSE of the pipeline without IMU data is 5.2 m versus 3.38 m for the full pipeline — approximately 54% higher.

• Ablation results (Table II, per-sequence): The full pipeline's best absolute-error configuration gives MH01 0.963 m / 26.96%, MH02 0.920 m / 28.86%, MH03 3.685 m / 18.61%, MH04 8.334 m / 48.77%, and MH05 3.376 m / 28.29% (absolute [m] / relative [%]). The best relative-error configuration gives MH01 1.115 m / 23.83%, MH02 4.431 m / 19.36%, MH03 2.843 m / 14.89%, MH04 10.781 m / 57.98%, and MH05 3.467 m / 29.96%. Removing IMU data gives MH01 6.092 m / 48.14%, MH02 4.995 m / 50.77%, MH03 2.572 m / 10.42%, MH04 81.374 m / 282.09%, and MH05 5.202 m / 28.45%. Removing both IMU data and EPnP gives MH01 7.228 m / 51.23%, MH02 5.982 m / 41.77%, MH03 8.867 m / 26.10%, MH04 12.061 m / 64.10%, and MH05 7.601 m / 46.80%.

• RMSE versus relative error trade-off: The parameter sweep scatter plot indicates that RMSE can be increased in order to reduce the relative translation error, meaning the parameterization acts as a tuning knob depending on whether short-term or long-term consistency matters more for the application. For the full pipeline, the best absolute RMSE and the best relative translation error are achieved by different parameter sets; for the other two configurations, the best parameter sets for both metrics are identical.

• Real-time behavior on GAP9: On the first 1578 frames of the MH01 sequence, the 8-point algorithm is executed in 57.3% and EPnP in 42.7% of VO iterations. The average pipeline runtime is below 50 ms, allowing the target 20 FPS.

• Parallelization speedups (Table III, kCycles single-core to multi-core): Feature Extraction 21817 to 2926 (7.46x), Frame Matching 8112 to 1033 (7.85x), World Matching 16782 to 2128 (7.89x), EPnP RANSAC 57324 to 10899 (5.26x), 8-point RANSAC 50391 to 10323 (4.88x), and Triangulation 374 to 374 (1.00x — performed only for keyframes and not parallelized).

• Not reported in the available content: The provided text is truncated mid-discussion of Figure 4 and Table III. The detailed memory utilization numbers for L1 and L2 (peak utilization per pipeline segment), the direct head-to-head accuracy comparison against other published systems on the datasets, and any power breakdown by pipeline segment are not present in the content supplied, so no values for them are stated here.

Methodology in Plain English

The researchers followed a constraint-driven design approach: rather than designing for accuracy first and squeezing the result onto hardware later, they let the limitations of the target hardware — the GAP9 SoC with its limited memory and multicore cluster — steer which algorithms went into the pipeline.

First, they built a Python "golden model" of the candidate pipeline. This let them rapidly test which pipeline components actually improved accuracy and robustness, and to sweep parameters (optimization window size, keyframe threshold, reprojection error noise model, IMU noise model, and gravity vector updates) before committing to a design. The sweeps were evaluated on EuRoC using the rpg_trajectory_evaluation tool, looking at absolute RMSE across all predicted poses and relative translation error over 40 m segments.

The selected design is indirect (feature-based): ORB detects and describes features; a two-way brute-force matcher pairs them; the 8-point essential-matrix computation with RANSAC estimates frame-to-frame rotation and translation, and EPnP with RANSAC estimates absolute camera pose against 3D landmarks. Keyframes are added only once the parallax relative to the previous keyframe exceeds a threshold. IMU measurements are pre-integrated between keyframes so that each keyframe interval yields a single relative motion constraint, and bundle adjustment of visual estimates is fused with those inertial constraints in one tightly coupled pose-graph optimization over a moving window.

Then came the embedded rewrite. The authors rewrote the pipeline in C with several adaptations: they had to write their own linear algebra library because libraries such as LAPACK could not run on GAP9 and no compact C library offered operations like SVD or the Jacobi Eigenvalue algorithm; they added a memory handler that keeps the pose graph and previous-keyframe data in L2 memory and stages it into L1 only when needed, treating L1 as a scratchpad with nothing preserved between iterations; they cut images to QQVGA to fit the 128 kB L1; they restructured the Levenberg–Marquardt optimization using the Schur complement to marginalize 3D landmark states and solve only for camera pose states; and they moved the optimizer onto the fabric controller so it runs continuously and largely independently of the VO iterations. Finally, they parallelized ORB detection and description, brute-force matching, and both RANSAC loops across the worker cores. The evaluated pipeline is monocular, and loop closure is omitted because of the memory limit.

Why This Matters

Impact on research: LEVIO demonstrates that a full six-DoF VIO pipeline with inertial fusion and bundle adjustment can run on a commercial ultra-low-power multicore SoC below 100 mW, rather than requiring a custom ASIC or a Linux-class board drawing single-digit watts. That shifts the reference point for what counts as a "resource-constrained VIO" baseline and makes the full pipeline architecture, not just a stripped-down variant, available for comparison.

Real-world applications:

• Micro- and nano-drones, where payload limits prevent carrying the compute platforms and battery packs required by conventional VIO systems.
• Augmented reality glasses, which need infrastructure-less, accurate movement tracking within a tight power and thermal budget.
• Mobile robotics operating indoors and outdoors without installed infrastructure such as motion-capture setups, Wi-Fi localization, or ultra-wideband anchors.
• Any battery-powered smart device that needs six-DoF motion sensing continuously rather than in short bursts.

Industry relevance: The complete implementation is open-sourced and written in C for commercial off-the-shelf hardware, which lowers the barrier to integration. The paper's comparison table frames LEVIO against implementations spanning 2 mW (a custom ASIC, Navion) up to 5.4 W (a transformer-based system on an NVIDIA Jetson Xavier NX), and against FPGA-assisted and on-sensor-accelerated designs, positioning it as a middle ground that requires neither custom silicon nor Linux-class compute.

Future Directions

• Closing the accuracy gap to server-class VIO. LEVIO deliberately omits loop closure due to memory constraints; quantifying what loop closure would cost in memory and power on this class of hardware, and whether a reduced form is affordable, is a natural next step. The paper's own stated aim is to bridge simplified embedded algorithms and accurate state-of-the-art ones, so measuring the remaining distance matters.

• Recovering the lost headroom from the ablations. The pipeline without IMU data and without EPnP is substantially less accurate, and even full LEVIO's MH04 result (8.334 m best absolute, 10.781 m best relative) stands out as difficult, suggesting robustness under aggressive motion or lighting remains an open problem worth investigating.

• Extending beyond the single validated platform. LEVIO was validated on GAP9, a nine-core cluster (master plus eight worker cores) with 128 kB L1 and 1.6 MB L2 running at up to 370 MHz. How well the design maps to other microcontrollers and SoCs with different memory hierarchies and core counts is not established in the provided content.

• Exploring the RMSE versus relative-error trade-off per application. The authors note that the parameterization lets RMSE be traded against relative translation error; determining which operating point suits which robotic or AR task, and whether that choice can be made adaptively at runtime, is left open.

Target Audience

This paper is most useful to embedded systems and robotics engineers building motion-tracking products on power-constrained hardware, researchers working on low-power VIO and SLAM who need a realistic power/accuracy operating point, and practitioners evaluating whether a full inertial pipeline — rather than a reduced optical-flow or planar approach — is feasible on microcontrollers and low-power multicore SoCs. Readers should have a working understanding of visual odometry and inertial fusion to follow the pipeline and optimization sections, though the problem framing and validation results are readable at a higher level.

Authors’ abstract

Accurate, infrastructure-less sensor systems for motion tracking are essential for mobile robotics and augmented reality (AR) applications. The most popular state-of-the-art visual-inertial odometry (VIO) systems, however, are too computationally demanding for resource-constrained hardware, such as micro-drones and smart glasses. This work presents LEVIO, a fully featured VIO pipeline optimized for ultra-low-power compute platforms, allowing six-degrees-of-freedom (DoF) real-time sensing. LEVIO incorporates established VIO components such as Oriented FAST and Rotated BRIEF (ORB) feature tracking and bundle adjustment, while emphasizing a computationally efficient architecture with parallelization and low memory usage to suit embedded microcontrollers and low-power systems-on-chip (SoCs). The paper proposes and details the algorithmic design choices and the hardware-software co-optimization approach, and presents real-time performance on resource-constrained hardware. LEVIO is validated on a parallel-processing ultra-low-power RISC-V SoC, achieving 20 FPS while consuming less than 100 mW, and benchmarked against public VIO datasets, offering a compelling balance between efficiency and accuracy. To facilitate reproducibility and adoption, the complete implementation is released as open-source.