Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment

AI summary

Overview

Research area: Robotics — Vision-Language-Action (VLA) models that unify visual perception, natural-language instruction following, and low-level robot control.

Technical level: Advanced (assumes familiarity with vision-language models, diffusion/flow-matching action experts, and transformer cross-attention architectures).

Scope: The paper introduces Evo-1, a 0.77-billion-parameter VLA model that reaches state-of-the-art manipulation results on Meta-World, LIBERO, and RoboTwin, plus real-world tasks, without any large-scale robot-data pretraining.

What This Paper Is About

Existing VLA models are typically very large (often several billion parameters) and depend on long training runs over massive robot datasets such as OXE and DROID. This makes them expensive to train, slow to run in real time, and prone to degrading the pretrained vision-language backbone's semantic representations during end-to-end training, which hurts generalization.

The paper's goal is a lightweight VLA that keeps the perceptual/linguistic understanding of its vision-language backbone intact while still learning to generate good robot actions — and that does so with low-cost training and high inference speed on consumer-grade hardware, without robot-data pretraining.

Key Contributions

1. Lightweight and efficient architecture. Evo-1 has only 0.77B parameters, reducing training cost and improving inference speed for real-time deployment on consumer-grade GPUs. It combines an InternVL3-1B vision-language backbone, a cross-modulated diffusion transformer action expert, and an integration module.

2. Semantic preservation for improved generalization. A two-stage training paradigm (first freeze the backbone and train only the integration module plus action expert; then unfreeze and fine-tune everything) balances preserving the VLM's inherent multimodal understanding against adapting it to action generation.

3. Strong performance without robot-data pretraining. State-of-the-art results on Meta-World (80.6%), RoboTwin (37.8%), competitive LIBERO results (94.8%), and a 78% real-world success rate — all without large-scale robot-data pretraining.

4. Open release. Code, data, and model weights are released to support future work on lightweight, efficient VLA models.

Main Findings

• Meta-World state of the art: Evo-1 reaches an average success rate of 80.6%, compared with 68.2% for SmolVLA (2.25B params) and 47.9% for π₀ (3.5B params). By difficulty level Evo-1 scores 89.2 (easy), 76.8 (medium), 77.2 (hard), and 79.2 (very hard). The abstract reports this as surpassing the previous best models by 12.4%.

• RoboTwin state of the art: Evo-1 attains an average of 37.8%, surpassing the previous best of 30.9% (π₀). Per task (easy | hard): Click Alarmclock 77.0 | 58.0, Dump Bin Bigbin 74.0 | 37.0, Place Bread Basket 15.0 | 3.0, Place Can Basket 37.0 | 1.0. The abstract reports this as a 6.9% improvement over the previous best.

• Competitive LIBERO result: Evo-1 scores 94.8% average — spatial 92.7, object 97.7, goal 96.3, long 92.3 — above π₀ (94.2%) and SmolVLA (88.8%), with notably strong robustness on long-horizon tasks (92.3%).

• Real-world success: Across four real-world tasks on a 6-DoF xArm6 with a parallel gripper, Evo-1 achieves 78% average success, versus SmolVLA (50%), OpenVLA-OFT (55%), and π₀ (73%), despite being roughly one-fourth the size of the 3.5B-parameter π₀.

• Best efficiency/performance trade-off: On an RTX 4090d GPU, Evo-1 uses 2.3 GB of GPU memory, runs at 16.4 Hz, and achieves 78% success — compared with SmolVLA (0.45B, 2.0 GB, 12.7 Hz, 50.0%), OpenVLA (7.0B, 15.1 GB, 7.9 Hz, 55.0%), and π₀ (3.5B, 17.9 GB, 11.5 Hz, 73.0%).

• Preserved semantic attention: Visualizations show that Evo-1's InternVL3-1B backbone keeps spatially consistent, semantically aligned attention maps after robot training, whereas OpenVLA's Prismatic-7B backbone shows degraded coherence and semantic drift.

• Two-stage training beats single-stage: The two-stage paradigm outperforms a single-stage baseline that trains all modules jointly from scratch on every Meta-World difficulty level, and attention maps show the single-stage variant loses clear semantic focus.

• Module A wins the integration ablation: Of four integration designs (A–D) evaluated on LIBERO-Long, the mid-layer cross-attention design (Module A, using the 14th VLM layer, concatenated with robot state as key-value input to all DiT layers) performs best because it propagates multimodal information consistently.

• Strong generalization under disturbance: In real-world "Pick and Place Can" tests with 20 trials per condition, Evo-1 beats SmolVLA throughout — base 95% vs 75%; unseen distractor object 80% vs 65%; background color change 75% vs 60%; target position shifted 10/20/30 mm backward 95%/85%/80% vs 75%/60%/60%; target height 10/20/30 mm higher 100%/90%/70% vs 75%/65%/60%.

Methodology in Plain English

Evo-1 is built from three pieces:

1. A vision-language backbone. The team uses InternVL3-1B, which was pretrained natively on both images and text rather than by bolting a vision encoder onto a text-only model. Its visual encoder is InternViT-300M (distilled from InternViT-6B), and its language branch is Qwen2.5-0.5B. Input images are resized to 448×448 and passed through pixel-unshuffle downsampling, cutting visual tokens by 4×. Only the first 14 layers of the language branch are kept, since intermediate layers were found to align visual and linguistic features better for visuomotor control.

2. An action expert. Actions are generated by a Diffusion Transformer that uses only stacked cross-attention layers (unlike prior VLA models that alternate self-attention and cross-attention). It follows the flow-matching paradigm: noisy actions are made by interpolating between ground-truth actions and random noise (Eq. 3), with the interpolation weight τ sampled from a Beta distribution and clamped to [0.02, 0.98] for numerical stability. The network learns a time-conditioned velocity field that pushes interpolated actions toward the ground truth (Eq. 4). The model configuration uses 8 transformer layers with dropout 0.2.

3. An integration module. The fused vision-language representation from the 14th backbone layer is concatenated with the robot's proprioceptive state (rather than projected into a shared space), and this concatenation serves as key-value input to the DiT's cross-attention layers, with the noisy action as the query.

Training happens in two stages. In Stage 1 the entire VLM backbone is frozen and only the integration module and action expert are trained, letting the randomly initialized action weights align with the multimodal embedding space without noisy gradients corrupting the pretrained backbone. In Stage 2, the backbone is unfrozen and the whole model is fine-tuned jointly.

Experiment setup: All experiments use distributed training on 8× NVIDIA A100 GPUs. In Meta-World runs, Stage 1 trains for 10k steps and Stage 2 for 65k steps, with learning rate 1×10⁻⁵, batch size 16, warmup 1k steps, gradient clipping 1.0, and weight decay 0.001 (other benchmarks follow the same scheme with minor step adjustments). Data follows the LeRobot v2.1 format; images are resized to 448×448; state and action vectors are padded to 24 dimensions; and the action horizon is H = 50. Meta-World uses 50 tasks with 50 demonstrations each (2,500 total), 10 trials per task, and 5 independent runs; LIBERO has 40 tasks in four categories, also with 10 trials and 5 runs; RoboTwin covers 4 tasks with 50 training demonstrations and 100 evaluation trials per task at two difficulty levels. Real-world evaluation uses 100 teleoperation demonstrations per task and 20 trials per task.

Why This Matters

Impact on research: The work challenges the assumption that competitive VLA performance requires billions of parameters and large-scale robot-data pretraining. It also provides evidence — via attention-map comparisons — that end-to-end training can erode a VLM's semantic representations, and that a staged training schedule can prevent this. The released code, data, and weights lower the barrier to reproducing and extending lightweight VLA research.

Real-world applications:

• Consumer-grade robot deployment: Running at 16.4 Hz with 2.3 GB of GPU memory makes real-time control feasible on modest hardware rather than data-center GPUs.
• Assistive and service robotics: The "Hand Delivery" task — placing a can into a human hand held at varying locations — points toward robots that interact safely with people.
• Household and kitchen manipulation: The "Pour Foam from Cup" and "Can Stacking" tasks reflect common everyday object-handling scenarios.
• Dual-arm industrial/warehouse tasks: RoboTwin tasks such as dumping bins and placing objects in baskets demonstrate bimanual coordination relevant to logistics.

Industry relevance: Eliminating robot-data pretraining removes a major cost center (data collection is described as labor-intensive and costly), and low memory/parameter requirements reduce hardware procurement costs. High inference frequency supports interactive applications where control latency matters.

Future Directions

• Scaling and generalization beyond the tested tasks. Evo-1 is evaluated on four real-world tasks and three simulation benchmarks; whether the architecture extends to longer-horizon or more contact-rich manipulations is not reported.
• Understanding the size/performance frontier. The paper shows Evo-1 outperforming much larger models, but does not report an analysis of how far the architecture scales up or down before performance degrades.
• Extending the two-stage recipe to other backbones. The semantic-preservation finding is demonstrated with InternVL3-1B versus Prismatic-7B; testing the schedule across more vision-language backbones is a natural next question.
• Broadening robustness evaluation. Generalization tests cover four disturbance types (distractor object, background color, target position, target height) in a single real-world task; the paper does not report testing other failure modes such as new embodiments or lighting changes.

Target Audience

Robotics and embodied-AI researchers working on VLA models and robot manipulation policies; efficiency-focused machine learning engineers interested in running policies on consumer GPUs; engineers evaluating deployment cost for real-world robot systems; and students with a background in transformers and diffusion models who want a concrete example of a lightweight, training-efficient multimodal control architecture.

Authors’ abstract

Vision-Language-Action (VLA) models have emerged as a powerful framework that unifies perception, language, and control, enabling robots to perform diverse tasks through multimodal understanding. However, current VLA models typically contain massive parameters and rely heavily on large-scale robot data pretraining, leading to high computational costs during training, as well as limited deployability for real-time inference. Moreover, most training paradigms often degrade the perceptual representations of the vision-language backbone, resulting in overfitting and poor generalization to downstream tasks. In this work, we present Evo-1, a lightweight VLA model that reduces computation and improves deployment efficiency, while maintaining strong performance without pretraining on robot data. Evo-1 builds on a native multimodal Vision-Language model (VLM), incorporating a novel cross-modulated diffusion transformer along with an optimized integration module, together forming an effective architecture. We further introduce a two-stage training paradigm that progressively aligns action with perception, preserving the representations of the VLM. Notably, with only 0.77 billion parameters, Evo-1 achieves state-of-the-art results on the Meta-World and RoboTwin suite, surpassing the previous best models by 12.4% and 6.9%, respectively, and also attains a competitive result of 94.8% on LIBERO. In real-world evaluations, Evo-1 attains a 78% success rate with high inference frequency and low memory overhead, outperforming all baseline methods. We release code, data, and model weights to facilitate future research on lightweight and efficient VLA models.