TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

AI summary

Overview

Research area: Robot learning / Vision-Language-Action (VLA) models for language-conditioned manipulation, with a focus on inference efficiency and real-time deployment.

Technical level: Advanced. The paper assumes familiarity with transformer architectures, cross-attention, behavior cloning, action chunking, and benchmark evaluation protocols for robotic manipulation.

Scope: The paper proposes TurboVLA, a VLA architecture that removes the large language model from the low-level control pathway and instead directly fuses visual and instruction features to predict continuous action chunks, evaluated on LIBERO, RoboTwin 2.0, and a real AgileX Piper robot.

What This Paper Is About

Most VLA models follow an LLM-centric V → L → A pathway: visual observations are projected into a large language model's representation space, combined with the instruction, and then decoded into robot actions. This makes every control step expensive in computation and memory, which limits control frequency and deployment on resource-constrained robots. The paper asks whether execution-level control can instead use a direct V + L → A mapping, where vision and language are encoded separately, interact directly through lightweight cross-attention, and are decoded into a chunk of continuous actions in one forward pass.

Key Contributions

1. A re-examination of LLM-centric VLA design. The authors identify the large language model core, rather than the action-generation mechanism, as the main bottleneck for real-time action execution, and propose a paradigm that keeps language conditioning while removing the LLM from execution-level control.

2. The TurboVLA architecture (V + L → A), combining lightweight instruction encoding with a compact text encoder, direct bidirectional vision-language interaction, robot-state conditioning, and non-autoregressive continuous action chunk prediction.

3. A strong performance-efficiency result on LIBERO: 97.7% average success with 0.2B parameters, 31.2 ms latency, and 0.9 GB inference VRAM on a consumer-grade RTX 4090, corresponding to more than 30 action chunk predictions per second (32 Hz).

4. Evidence beyond LIBERO, including bimanual manipulation on RoboTwin 2.0 (60.2% average success across 50 tasks) and real-world deployment on an AgileX Piper platform, where TurboVLA reports 92.5%, 80%, 90%, and 87.5% success on four tasks and consistently outperforms π0.5 under the same platform, training data, and evaluation protocol.

Main Findings

• LIBERO performance-efficiency frontier: TurboVLA reaches 97.7% average success (LIBERO-Spatial 99.2, LIBERO-Object 99.8, LIBERO-Goal 97.4, LIBERO-Long 94.2), versus 96.9% for π0.5, while using about 6% of π0.5's parameter count and reducing latency from 93.6 ms to 31.2 ms. It also outperforms VLA-JEPA (97.2% average success) in average success while being over 3× faster and using only about 7% of its parameters.

• Memory and scale: TurboVLA uses 0.9 GB inference VRAM and 0.2B parameters, compared with 12.8 GB / 3.4B for π0.5, 15.7 GB / 7.7B for OpenVLA-OFT, and 16.1 GB / 8.3B for CogVLA. The reported parameter count for LIBERO corresponds to the DINOv3 ViT-B configuration.

• Versus acceleration-oriented VLAs: OpenVLA-OFT and Discrete Diffusion VLA report latencies of 112.2 ms and 60.8 ms respectively, yet the paper states both remain slower and yield lower average success because their large language backbones are retained at the center of execution.

• Versus lightweight VLAs: TurboVLA improves both sides of the trade-off relative to Evo-1 (94.8% average success, 0.8B parameters, 1.7 GB, 137.2 ms) and VLA-Adapter (97.3%, 1.5B, 4.3 GB, 87.3 ms), while being substantially smaller and faster.

• Bimanual manipulation: On RoboTwin 2.0, TurboVLA achieves 60.2% average success across 50 tasks with 43.4 ms latency and 0.4B parameters (DINOv3 ViT-L configuration), outperforming π0.5 (57.0%, 95.6 ms) and StarVLA-α (50.3%, 74.9 ms).

• Real-world deployment: On the AgileX Piper platform, TurboVLA achieves 92.5%, 80%, 90%, and 87.5% success on four tasks (grab roller, move playing card away, press stapler, stack three bowls), consistently outperforming π0.5; each task is evaluated over 40 trials.

• Language matters: Removing language drops average LIBERO success from 97.7% to 70.8%, with the largest drop on LIBERO-Goal (97.4% → 11.6%). Replacing semantic instructions with a learned task-ID embedding reaches 95.4%, still 2.3% below the full model.

• Text encoder choice is not critical: T5-Small attains 97.1% average success (141.9M overall parameters), SigLIP-Base 95.5% (216.9M), and BERT 97.7% (216.1M).

• Bidirectional interaction is best: No interaction (direct concatenation) gives 95.2% average success, "Language Queries Visual" 96.1%, "Visual Queries Language" 96.5%, and bidirectional interaction 97.7%.

• Depth and horizon: Average success rises from 93.5% at N=2 to 95.7% at N=4 and 97.7% at N=6, then slightly degrades to 96.6% at N=8. Performance improves from 96.4% at H=8 to 97.7% at H=12, then drops to 95.6% at H=15.

• Stated limitation: TurboVLA is designed primarily for concrete execution-level instructions and may not provide the complex semantic understanding and reasoning required for high-level task planning.

Methodology in Plain English

The authors keep the two input modalities separate instead of merging them inside a language model. A visual encoder (DINOv3) extracts spatial features from camera observations, with positional and camera-view embeddings added, and the features from K camera streams are concatenated. A lightweight text encoder (BERT) produces token-level instruction features, and the full token sequence is retained rather than pooled, so objects, attributes, and spatial relations stay available for visual conditioning. Both sets of features are projected into a shared dimension of d=256.

The two feature streams are then updated by N=6 stacked bidirectional cross-attention layers, initialized from grounding-pretrained feature-enhancement weights. Visual-to-instruction attention injects scene context into the instruction stream, and instruction-to-visual attention conditions visual features on the task semantics. The final visual and instruction streams are concatenated.

Robot state is not passed through the interaction module; it is encoded separately with a lightweight projection network and introduced only at the action decoder. An ACT-style lightweight transformer decoder takes a set of H learnable action queries and cross-attends to the fused multimodal features plus the state features, producing all H actions in parallel in a single forward pass, with no action tokenization or sequential generation. Training is behavior cloning with an ℓ1 loss and no auxiliary language-modeling objective, at a learning rate of 5×10⁻⁵ on four RTX 4090 GPUs.

For LIBERO, one mixed-suite model with a DINOv3 ViT-B backbone predicts 12-step chunks of continuous 7-DoF actions, trained for 80k steps with 10k warm-up steps and an effective batch size of 256, using the modified no_noops RLDS datasets released with OpenVLA; evaluation follows the VLA-Adapter rollout protocol with 50 rollouts per task over 2,000 trials. For RoboTwin 2.0, one multi-task model with a DINOv3 ViT-L backbone predicts 50-step chunks of 14-dimensional absolute joint-position actions, trained for 55k steps with 1k warm-up steps and an effective batch size of 192 on official clean demonstrations only, evaluated with 100 clean-setting rollouts per task following the StarVLA framework. For real-world experiments, the policy is initialized from the LIBERO-pretrained checkpoint and fine-tuned on 4×65 teleoperated demonstrations for 12.5k steps.

Why This Matters

Impact on research: The paper challenges a widely held assumption in the VLA literature — that broad multimodal pretraining must sit at the center of the control loop. It suggests that execution-level control only needs enough language understanding to determine how current visual evidence should guide action, and that the community should evaluate VLA systems beyond task success alone, on latency and memory as well.

Real-world applications:

• Deploying language-conditioned manipulation policies on robots with consumer-grade or embedded GPUs, where multi-gigabyte inference footprints are impractical.
• Latency-sensitive closed-loop control such as responsive pick-and-place or reactive manipulation, where 32 Hz action chunk prediction supports tighter feedback.
• Bimanual manipulation platforms that require coordinated dual-arm control, as demonstrated on the 50 RoboTwin 2.0 tasks.
• Lowering the hardware barrier for research labs and small companies that want to run VLA policies without high-memory GPU clusters.

Industry relevance: The reported 0.9 GB inference VRAM and 31.2 ms latency position TurboVLA as a candidate for cost-constrained commercial robotics where per-unit GPU memory and control frequency directly affect system design. The paper's framing also implies that system-level compression and optimization may be less necessary if the architecture itself avoids the large generative backbone.

Future Directions

• Hierarchical systems: The authors state that future work will explore combining the high-level planning capability of LLMs with TurboVLA's efficient execution pathway, building hierarchical systems that are both intelligent and efficient.

• Beyond concrete execution-level instructions: Since TurboVLA is designed primarily for concrete execution-level instructions, an open question is how it handles complex semantic understanding, open-ended reasoning, or high-level task decomposition.

• Generalization and data scale: Training on RoboTwin 2.0 was restricted to official clean demonstrations without randomized-scene data; whether the approach extends to randomized scenes and broader cross-embodiment data is not established in this paper.

• Broader benchmarking: The evaluation covers LIBERO, RoboTwin 2.0, and four real-world tasks on one platform. How the direct V + L → A paradigm behaves on other embodiments, longer-horizon tasks, and open-vocabulary settings is not reported.

Target Audience

Robotics and embodied-AI researchers working on VLA models and language-conditioned manipulation, particularly those interested in inference efficiency, real-time control, and architectural alternatives to LLM-centric policies. It is also relevant to practitioners deploying manipulation policies on hardware with limited GPU memory, and to readers tracking the debate over how much of a large language model is actually needed for low-level robot control.

Authors’ abstract

Vision-language-action (VLA) models commonly adopt an LLM-centric $V \to L \to A$ pathway, where visual observations are projected into the representation space of a large language model before being decoded into robot actions. Although effective, this design incurs substantial computation and memory overhead at every policy invocation. In this work, we introduce TurboVLA, a new VLA paradigm that reformulates the conventional $V \to L \to A$ pathway as a direct $V + L \to A$ mapping. Instead of using a large language model as the central interface between perception and action, TurboVLA independently encodes visual observations and language instructions, directly exchanges information between them through lightweight bidirectional vision-language interaction, and predicts continuous action chunks with a compact decoder. This simple design constructs task-conditioned representations directly from visual and linguistic features, significantly reducing the computational and memory costs of VLA inference. On LIBERO, TurboVLA achieves 97.7% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GB inference VRAM on a consumer-grade RTX 4090, matching or outperforming substantially larger VLA policies. These results establish TurboVLA as a simple and effective alternative to the prevailing LLM-centric VLA paradigm, offering a new perspective on how vision, language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H-EmbodVis/TurboVLA.