MARC: Memory-Augmented RL Token Compression for Efficient Video Understanding

AI summary

Overview

• Research area: Computer vision / multimodal video understanding, specifically visual token compression for vision-language models (VLMs).
• Technical level: Advanced. The paper assumes familiarity with vision-language models, reinforcement learning (GRPO/PPO-style objectives), and token merging/compression literature.
• Scope: The paper proposes and evaluates MARC, a retrieve-then-compress framework combining a Visual Memory Retriever with a new RL post-training objective (C-GRPO) to reduce video token counts to roughly one frame's worth while preserving accuracy.

What This Paper Is About

Video-language models process many frames, and each frame becomes many visual tokens, so inference becomes expensive and slow for high-frame-rate or long-duration video. Existing token-compression methods are mostly training-free merging or pruning schemes that cut cost but discard information and lose accuracy. MARC attacks the problem by first retrieving only the query-relevant event segments of a video (a Visual Memory Retriever), then training the model with a reinforcement-learning objective (Compression Group Relative Policy Optimization) that pushes a heavily compressed "student" to retain the reasoning performance of a 64-frame "teacher."

Key Contributions

1. MARC framework: A retrieve-then-compress pipeline for efficient video understanding that pairs structured visual retrieval with reinforcement-learning-based token compression.
2. Compression Group Relative Policy Optimization (C-GRPO): Described by the authors as the first post-training RL strategy designed specifically for video token compression, transferring reasoning ability from a 64-frame teacher network to a 1-frame student network via a retention-alignment reward.
3. Visual Memory Retriever (VMR): An event-based segmentation and nearest-neighbor retrieval module, motivated by cognitive science and neuroimaging findings on episodic memory and event segmentation, that partitions video into semantically coherent clips and returns the top-k query-relevant fragments.
4. Empirical validation across six benchmarks: MARC-3B matches the 64-frame Qwen2.5-VL-3B baseline mean (42.20 vs. 42.21) with a single frame's worth of tokens, while reducing GPU memory usage by 72% and generation latency by 23.9%.

Main Findings

• Accuracy parity at extreme compression: On six benchmarks (VSI-Bench, VideoMMMU, MMVU (mc), MVBench, TempCompass, VideoMME w/o sub), MARC-3B scores a mean of 42.20 using 1 frame versus 42.21 for the 64-frame Qwen2.5-VL-3B baseline, using 4.71% of the original visual tokens.
• Token reduction: Average visual tokens per sample drop from 2589.93 (64 frames) to 122.69 (1 frame), a 95% reduction.
• Per-benchmark results for MARC-3B: VSI-Bench 27.55, VideoMMMU 33.11, MMVU (mc) 51.99, MVBench 45.82, TempCompass 55.34, VideoMME (w/o sub) 39.44. Baseline 64-frame scores are 32.93, 35.33, 48.64, 44.77, 38.05, and 53.55 respectively.
• Gains on specific benchmarks: MARC-3B outperforms the baseline on MMVU, MVBench, and TempCompass; the largest improvement is on TempCompass (55.34 vs. 38.05), attributed to improved instruction-following from the training process.
• Beats larger models on average: MARC-3B's mean of 42.20 exceeds InternVL3.5-4B (42.00), Gemma-3-4B (38.87), and InternVL3.5-2B (15.95).
• Outperforms prior compression methods: MARC improves mean accuracy by 65%, 31%, and 52% over DynamicVLM, MovieChat, and VidCom respectively. Table 1 lists ByteVideoLLM-3B at 25.52, MovieChat-3B (1 frame) at 32.10, and VidCom2-3B at 27.84.
• Efficiency: Baseline Qwen2.5-VL with 64 frames occupies 41.63 GB of 48 GB GPU memory with a batch size of 15; MARC reduces this to 11.48 GB, a 72.4% reduction. Generation latency falls 23.9%, overall model generation latency falls 15.9%, and end-to-end latency for a single MMVU sample falls 11.1%.
• Long-video limitation: On VideoMME, MARC retains 74% of baseline performance while using 3.21% of the original visual tokens. The paper attributes this to long videos being split into many clips, where restricting retrieval to the top-3 discards critical information; VideoMME medium videos average 515 seconds and long videos average 2466 seconds, versus 506 seconds for the longest other benchmark (VideoMMMU).
• VMR alone helps: Untrained Qwen2.5-VL-3B with VMR (top-k = 3, 64 frames) reaches a mean of 45.56 vs. 42.21 for the baseline, with an increase as high as 27.85% on MVBench. VMR hurts VideoMME (53.55 to 51.85) and slightly hurts VideoMMMU.
• C-GRPO beats SFT under compression: MARC-3B-0.6 (42.20 mean) exceeds an SFT baseline trained on 10K samples from Video-R1-COT-165K (38.50 mean), a relative gain of +9.6%, and exceeds an SFT+VMR variant (40.16 mean) by 5%. SFT+VMR improves the mean over SFT by 4.3%, with gains above 10% on VideoMMMU and MVBench.
• Threshold sensitivity: Ablating the retention threshold tau gives means of 41.34 (tau = 0.4), 42.20 (tau = 0.6), and 41.40 (tau = 0.8); tau = 0.6 is selected as the balanced choice.

Methodology in Plain English

MARC works in two stages.

Stage one: retrieve. The Visual Memory Retriever converts a continuous video into a structured memory bank. An event-detection network finds temporal boundaries such as scene changes, topic shifts, or the start of new actions, splitting the video into semantically coherent clips that each represent a complete event. Both the query and the fragments are embedded into a shared latent space learned with contrastive learning, and an optimized nearest-neighbor search returns the top-k most relevant fragments. This shrinks the search space and lets the model focus on the parts of the video that matter for the question.

Stage two: compress. The retrieved fragments are sampled at 1 fps to obtain N frames, and each frame is encoded by a visual encoder (e.g., ViT) into patch-level hidden states. These tokens are divided into short-term windows of m frames. Within each window, the two most similar temporally consecutive frames (measured by average patch-aligned cosine similarity) are repeatedly merged by averaging until the window hits its budget, computed from a compression ratio rho. If the total still exceeds the target, a lightweight cross-segment merge is applied as a last resort, so local event structure is preserved first.

Stage three: train with C-GRPO. The compression problem is treated as distillation. A teacher model processes 64 frames; a student processes the compressed single-frame input. C-GRPO defines a retention ratio eta as the compressed-input reward divided by the 64-frame reward, and adds a compression reward scaled by how much eta exceeds a threshold tau. That bonus is gated on answer correctness, so only semantically valid generations earn retention credit. Advantages are normalized within a group of size 8, and the policy is optimized with a clipped objective plus a KL anchor to a reference policy.

Experimental setup. Videos are sampled at 1 fps and capped at 64 frames, with frames capped at 128x28x28 resolution (14x14 patches). Decoding uses top_p = 0.001 and temperature = 0.01, max output length 1024, Flash Attention 2, on NVIDIA A6000 GPUs with 48 GB. The backbone is Qwen2.5-VL-3B. Training uses 5K randomly sampled instances from the Video-R1-260K dataset; only MARC-3B experiments use VMR with top-k = 3.

Why This Matters

Impact on research. The paper argues that training-free compression inherently loses information, and proposes an RL post-training alternative that aligns a compressed student to an uncompressed teacher. It frames compression as an alignment problem rather than a purely geometric token-reduction heuristic, and couples the compressor to the retrieval stage so compression operates within event boundaries rather than uniformly over the timeline.

Real-world applications (as named in the paper):

• Real-time video question answering in latency-sensitive settings.
• Surveillance systems, where computation is constrained and responses must be fast.
• Autonomous driving, where video streams are continuous and hardware budgets are tight.
• General deployment of VLMs on resource-constrained hardware, given the reported 72% GPU memory reduction.

Industry relevance. The headline efficiency numbers — 41.63 GB down to 11.48 GB, 23.9% lower generation latency, and a 95% token reduction at near-parity accuracy — target the practical bottleneck of serving video models. The paper positions this as enabling deployment where input videos are too large for current inference budgets.

Future Directions

• Handling very long videos. MARC retains only 74% of baseline performance on VideoMME. The paper notes that increasing top-k does not simply fix this, because long videos can produce more than 100 clips.
• Adaptive frame selection. The authors propose a temporal Q-Former that adaptively identifies key frames for compression as a potential mitigation, and leave it for further exploration.
• Retrieval policy for long streams. Restricting retrieval to top-3 discards information once videos are divided into many clips; how best to select and rank clips at scale remains open.
• Broader generality. The paper reports results on Qwen2.5-VL-3B, with baseline comparisons against InternVL3.5-2B, InternVL3.5-4B, and Gemma-3-4B, but does not report whether C-GRPO is applied to those backbones; extending the training recipe across backbones is an unresolved question.

Target Audience

This paper is most useful to researchers and engineers working on multimodal large language models, video understanding, and inference efficiency — particularly those interested in token compression, retrieval-augmented video pipelines, or reinforcement-learning post-training. Practitioners deploying video-capable VLMs on constrained hardware will find the efficiency results directly relevant. Readers without background in reinforcement learning or vision-language architectures will find the methodology section demanding, though the retrieve-then-compress intuition is accessible.

Authors’ abstract

The rapid progress of large language models (LLMs) has laid the foundation for multimodal models. However, visual language models (VLMs) still face heavy computational costs when extended from images to videos due to high frame rates and long durations. Token compression is a promising solution, yet most existing training-free methods cause information loss and performance degradation. To overcome this, we propose \textbf{Memory-Augmented Reinforcement Learning-based Token Compression (MARC)}, which integrates structured retrieval and RL-based distillation. MARC adopts a \textit{retrieve-then-compress} strategy using a \textbf{Visual Memory Retriever (VMR)} to select key clips and a \textbf{Compression Group Relative Policy Optimization (C-GRPO)} framework to distil reasoning ability from a teacher to a student model. Experiments on six video benchmarks show that MARC achieves near-baseline accuracy using only one frame's tokens -- reducing visual tokens by \textbf{95\%}, GPU memory by \textbf{72\%}, and latency by \textbf{23.9\%}. This demonstrates its potential for efficient, real-time video understanding in resource-constrained settings such as video QA, surveillance, and autonomous driving.