The Pulse
Google Adds Agentic Video Understanding to Gemini
Google DeepMind has added agentic video understanding to Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. The feature lets Gemini search video dynamically, cutting token use by up to 88% and analysis costs by up to 66%, according to Google.
AI.info Team ·
Google says its new agentic video feature cuts token consumption by up to 88%, lowers analysis costs by as much as 66% and improves accuracy by up to 7%. The capability, announced September 1, gives Gemini a way to search through video instead of processing every moment at a fixed sampling rate.
Agentic video understanding is available through the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform. Google is launching it across Gemini 3.7 Flash, Gemini 3.6 Flash and Gemini 3.5 Flash-Lite, with no additional feature fee beyond standard Gemini API token pricing. Google DeepMind’s announcement says the system works with uploaded videos and public YouTube videos.
88% fewer tokens for long-form video
Gemini’s standard video processing extracts frames at a fixed rate, with 1 frame per second as the default. Developers can adjust that rate, but a fixed sampling strategy creates a trade-off: processing more frames raises cost and token use, while processing fewer frames can miss fast movements, brief scene changes or small visual details.
Agentic processing changes the sequence. Gemini can decide which portions of a video deserve closer inspection, then request frames, audio or transcripts from those sections. Google says the approach is particularly useful for videos ranging from 10-minute tutorials to 90-minute lectures and recordings lasting several hours.
“Agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the moments and signals needed,” write Rohan Doshi, senior product manager at Google DeepMind, and Mario Lučić, research director at Google DeepMind, in the announcement.
Gemini chooses where to look
The system runs through an agentic loop. A user supplies a video and a question; Gemini identifies the evidence it needs, loads the relevant part of the video and can inspect it at a different frame rate or through another modality. The model can combine visual frames with audio and transcripts rather than treating the entire file as one uniformly sampled stream.
Google says that process supports several tasks that fixed-rate analysis can handle less precisely. Gemini can retrieve a moment at sub-second resolution, identify anomalies by resampling a short time window, count repeated movements or objects, and search long recordings for a specific event without loading the entire video into context.
The company’s benchmark claims apply across the three models announced for the feature, with Gemini 3.7 Flash delivering the strongest overall quality and the best tested balance between accuracy and cost. Google does not present the figures as a universal guarantee; the results depend on the video, prompt and model’s navigation choices.
Three Flash models get the feature
Developers can activate the capability by setting video processing to "agentic" in the API configuration. Google’s developer documentation shows the same approach for longer files uploaded through the Files API, as well as for YouTube URLs and other supported video inputs.
The Gemini API documentation describes agentic mode as a fit for long-form questions and searches for specific moments. Static processing can still be preferable for short clips when a developer needs frame-level coverage across the entire file or wants to minimize the extra navigation step before generation begins.
For multi-turn conversations, the API preserves the video context through interaction state or returned processing steps. Google’s documentation says developers must retain those steps in stateless follow-up requests; omitting them can cause Gemini to lose the context and reprocess the video.
From API calls to YouTube answers
Google is positioning the feature as infrastructure for applications that need to reason over video rather than simply summarize it. Potential uses include searching recorded meetings, analyzing instructional footage, reviewing sports sequences, detecting unusual activity and building video-powered agents that answer questions about specific moments.
The initial release targets developers through Google AI Studio and Gemini Enterprise Agent Platform. Google also says it plans to bring the efficiency and quality improvements to users of the Gemini app across Flash and Flash-Lite models.
In the coming months, Google says agentic video understanding will also support YouTube’s “Ask YouTube” feature on video watch pages. That rollout would move the technology from an API setting into a consumer-facing question-and-answer system grounded in the visual content of videos.
The trade-off Google leaves to developers
Agentic processing does not eliminate the need to choose a mode. Dynamic inspection can reduce the amount of video Gemini reads, but it introduces an additional reasoning and tool-use step before the final answer. For short clips or tasks that require examining every frame, Google’s own documentation continues to describe static processing as the better fit.
The immediate change is narrower and more practical: Gemini no longer has to treat a long video as a fixed stream of equally important frames. Developers can ask for an answer, and the model can decide which seconds, signals and sampling rates matter before producing it.