The Pulse
NVIDIA’s 120B Nemotron 3.5 Super VL is available on SageMaker
AWS has made NVIDIA’s Nemotron 3.5 Super VL vision-language model available through SageMaker JumpStart. The service offers an NVFP4 version for optimized inference and BF16 reference weights, with different supported instances and reported
AI.info Team · Written with AI · fact-checked ·
A vision-language model joins JumpStart
AWS says NVIDIA’s Nemotron 3.5 Super VL is available through SageMaker JumpStart. The model takes images, video and text as input and produces text, and AWS says it can be deployed through SageMaker Studio or the SageMaker Python SDK.
At its core, the model combines a 120B-parameter language backbone with 12B active parameters and a C-RADIO vision encoder. The backbone has 88 layers, including Mamba-2 and MoE layers, with 512 routed experts and 22 active per token, plus eight attention layers. NVIDIA describes the model as aimed at document intelligence, graphical-interface understanding for computer-use agents, video question answering and multimodal agentic applications.
Two versions, different hardware
JumpStart offers two versions: NVFP4, intended for optimized inference, and BF16 reference weights for customization. AWS lists the NVFP4 checkpoint at about 75 GB and the BF16 checkpoint at about 249 GB. The NVFP4 option runs on G7 instances, while BF16 is supported on ml.p5en.48xlarge.
The model supports context lengths of up to 1 million tokens, although JumpStart serving configurations use between 520 and 15,600 tokens. For document tasks, AWS says a 1,536 x 1,024 page image uses about 1,536 context tokens; the serving configuration accepts one image and one video per request. The endpoint accepts images rather than raw PDF files, so users need to render PDF pages as images before sending them.
AWS reports that NVFP4 matched BF16 on MMLongBench-Doc, scoring 62.82 for both, and stayed within about one point on the other listed multimodal benchmarks. NVFP4 scored 68.34 on OCRBench v2, compared with 68.12 for BF16; on OSWorld-Verified, the scores were 52.30 and 51.14, respectively.
Performance depends on configuration
In AWS’s inference test, NVFP4 delivered mean output throughput of 63.07 tokens per second per request on ml.g7.24xlarge and 64.01 on ml.g7.48xlarge. The BF16 version on ml.p5en.48xlarge produced 140.85 output tokens per second per request, with a mean time to first token of 0.53 seconds. AWS cautions that these results are not a direct comparison of BF16 and NVFP4 precision because the variants ran on different GPU systems and configurations.
The test used 250 input tokens, 1,500 output tokens and five concurrent streaming requests. AWS measured a 12.62-second time to first token on ml.g7.24xlarge with the summarize profile, which used 10,000 input tokens. The company says users can deploy from Hugging Face as well as through AWS’s tools; deployment takes about 10 minutes on ml.g7.24xlarge, and new accounts start with G7 service quotas at zero.