The Pulse
NVIDIA Releases 4GB Nemotron Model for Local Agents
NVIDIA released Nemotron 3.5 Bolt E4B 4GB for local conversational agents, including assistants, interactive characters and embedded interfaces. The model has a 64k context window, and NVIDIA’s benchmark results show gains on some tasks but
AI.info Team · Written with AI · fact-checked ·
NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23, offering developers a small language model designed to run local agents on devices rather than send every request to a cloud service. The company targets voice and device assistants, interactive characters, and embedded interfaces, including robotics and industrial systems. The release follows NVIDIA’s October 15 announcement that the model would arrive on October 23.
Four gigabytes is a workload target, not the parameter count
The model is built on Google’s Gemma 4 E4B and further trained by NVIDIA for conversation, instruction following and multi-turn tool use. Its model card lists 8 billion parameters in total, with 4 billion effective parameters, and the downloadable 4GB version uses Q4_K_M quantization. NVIDIA says the model is designed to use about 4GB of memory in its target workloads; memory use varies with context length and runtime settings.
Its 64k context window accommodates conversation history, instructions, tool definitions, retrieved information and application state. NVIDIA lists 35 supported languages, building on a base model pretrained on more than 140. Its input and output are text, so the model does not itself process images. NVIDIA’s release page describes a robotics demonstration in which a separate Cosmos 3 Edge model supplies visual understanding while Bolt handles dialogue and calls scripts.
NVIDIA’s tests show gains—and uneven results
NVIDIA reports roughly 40% higher average quality than its earlier Nemotron 3 Nano 4B on selected conversational-agent benchmarks, and up to 80% fewer output tokens than peers in reasoning-enabled comparisons. Those are company-run evaluations, not independent results. The benchmark table also shows that performance varies by task: in its BF16 results, Bolt scores 53.1 on MultiChallenge, above Nano 4B’s 35.3, but its 55.1 Tau2-Bench score is below Qwen 3.5 4B’s 75.1.
NVIDIA cautions against treating any single benchmark as a proxy for performance in a real application. NVIDIA, the model’s developer, writes:
Current benchmarks only partially capture what makes these agents work well, and no individual benchmark covers every capability an application needs.
Its published hardware tests measure the quantized Q4_K_M model with llama.cpp at batch size one; the company says results can vary with hardware, inference software and configuration.
Local deployment comes with platform and license conditions
NVIDIA provides instructions for running Bolt through llama.cpp, and the model is also listed for use with tools including vLLM, SGLang, Ollama and Docker Model Runner. NVIDIA lists GeForce RTX, Jetson and RTX/DGX Spark systems among its target platforms. On an RTX 5070, the company reports 3.5GB of VRAM use, a 548-millisecond time to first token and 133.4 tokens per second for a workload with 3,000 new input tokens and 50 output tokens.
The model is available for commercial and non-commercial use under NVIDIA’s OpenMDW License Agreement, version 1.1; the underlying Gemma 4 model is governed by Apache 2.0. That makes the license terms part of deployment planning, alongside the hardware requirements. Bolt’s practical proposition is narrower than a general-purpose desktop chatbot: it is a text model tuned for short, interactive exchanges and tool use where keeping inference local matters.