FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Folding

AI summary

Overview

Research area: Efficient inference and post-training quantization for vision-language-action (VLA) robot policies, with deployment on NVIDIA Ada GPUs and Jetson AGX Orin.

Technical level: Advanced. The paper assumes familiarity with integer GEMMs, per-token/per-channel scale layouts, Hadamard rotations, GPTQ-style weight rounding, and TensorRT plugin authoring.

Scope: The paper introduces FoldQuantVLA, a post-training quantization framework that carries one consistent activation coordinate system through calibration, weight rounding, and native low-bit execution in both the language backbone and the iterative action expert of four policy checkpoints, evaluated in LIBERO, SimplerEnv, and on ALOHA and SO-101 robots.

What This Paper Is About

VLA policies combine visual processing, a language backbone, and an iterative denoising action expert, and both backbone and expert sit on the latency-critical path from observation to action. Naively dropping projections to four-bit weights and activations (W4A4) degrades action quality, and closed-loop behavior is especially exposed because action errors change the observations the policy sees next. The paper's goal is a post-training recipe requiring no policy retraining that makes low-bit execution actually fast at batch 1 while keeping robot behavior close to the floating-point reference.

Key Contributions

1. Consistent folding contract. One transformed coordinate system is assigned to each shared activation site, with the inverse folded into every consuming weight, and that same basis is carried through calibration, rounding, and native low-bit execution. Proposition 1 shows the transformed graph computes the same external function in exact arithmetic.
2. A W4A4 recipe for VLA policies. Channel scaling, block Hadamard rotations applied via the fast Walsh–Hadamard transform (FWHT, block size β = 64), and dynamic per-token quantization, avoiding timestep-indexed scale tables. Proposition 2 gives conditions under which normalized projection error is step-invariant.
3. Selective INT8 at o_proj and down_proj. Keeping language attention-output and feed-forward down projections at W8A8 while the rest remain W4A4 improves held-out action fidelity on all four checkpoints and raises observed GR00T N1.7 robot success from 80.0% to 92.5% for 1 ms of added Orin latency.
4. Deployable native low-bit engines. CUTLASS v3.6.0-based TensorRT plugins execute INT4 activations and W4A4 projections in both backbone and expert, giving 1.20–1.33× speedups over floating-point TensorRT on Orin, with evaluation spanning LIBERO, SimplerEnv, ALOHA, and SO-101.

Main Findings

• Latency gains over compiled float engines. W4A4 projections deliver 1.66–2.22× speedup over eager PyTorch and 1.25–1.52× over floating-point TensorRT on desktop, and 2.24–5.01× over eager PyTorch and 1.20–1.33× over the float engine on Orin. Compilation alone accounts for 83–92% of the eager-to-W4A4 reduction.
• W4A4 beats W8A8 on latency. W4A4 reduces latency by 10.3–15.3% relative to FQ W8A8.
• Weight-only quantization does not accelerate. ModelOpt W4

Authors’ abstract

Low-bit vision-language-action inference must reduce observation-to-action latency while preserving robot behavior. We present FoldQuantVLA, a post-training quantization framework that carries a consistent activation representation through calibration, weight rounding, and native integer execution. It combines channel scaling and block Hadamard transforms with dynamic per-token quantization, without policy retraining. Custom TensorRT plugins execute projections in both the language backbone and iterative action expert with four-bit weights and activations (W4A4) on Ada GPUs and Jetson AGX Orin. Evaluation spans LIBERO, SimplerEnv, and two robot platforms. Across three GR00T checkpoints and $π_{0.5}$, W4A4 achieves $1.20$ to $1.33\times$ speedups over floating-point TensorRT on Orin and $1.25$ to $1.52\times$ on desktop. Retaining language attention-output and feed-forward down projections at eight bits (W8A8) improves held-out action fidelity on all four checkpoints. Across four real-robot tasks, this configuration raises observed GR00T N1.7 success from $80.0\%$ with uniform W4A4 to $92.5\%$ over 80 trials per configuration, with a measured additional Orin latency of 1 ms.