The Pulse
NVIDIA Releases Dory, a 4B Model With Adjustable Inference Depth
NVIDIA’s Dory1.0-4B lets developers adjust inference depth by running a shared block of layers one to four times, trading compute for accuracy. Its benchmark results show gains on some coding tasks as loop count rises, while other scores fl
AI.info Team · Written with AI ·
NVIDIA’s Dory1.0-4B offers developers a choice: spend more inference compute on a response without loading a larger model, or keep the model to a single pass for lower compute per token. The company’s own benchmark table shows the trade-off is uneven. On SWE-bench Verified, Dory scores rise from 50.3 at one loop to 61.8 at four; on GPQA Diamond, the score rises from 69.19 to 72.41 at two loops, then slips to 72.16 at four.
NVIDIA dated the release October 16, 2026, on its Hugging Face model card. The checkpoint has 4 billion parameters and supports up to 262,144 tokens of context in NVIDIA’s BF16 launch configuration. Rather than increasing its parameter count, Dory repeatedly applies the same 20-layer block between an eight-layer input block and an eight-layer output block.
A shared block sets Dory’s depth
The model uses a looped, or latent-recurrent, normalized-GPT architecture with sliding-window and full-attention layers. At inference, developers can set the shared block to run one, two, three or four times. NVIDIA recommends two loops as a starting point, describing one loop as an option when reducing compute per token matters most and four as an option when additional accuracy is worth the cost, particularly for agentic tasks.
Repeated computation takes place in the model’s latent representations rather than requiring longer generated answers, according to the card. NVIDIA says looping can reduce the number of output tokens needed to solve a task, but also advises developers to evaluate end-to-end latency and cost on their own workloads. The benchmark scores alone do not establish whether extra loops make a particular application faster or cheaper overall.
Coding gains come with uneven scores
The clearest gain in NVIDIA’s table appears on coding and agent benchmarks. Dory’s SWE-bench Verified score climbs from 50.3 at one loop to 59.5 at two and 61.8 at four. At four loops, that is above the listed 54.7 for Qwen3.5-4B and 61.0 for Qwen3.5-9B.
Other results temper that comparison. On GPQA Diamond, Dory’s four-loop score of 72.16 trails the listed scores for Qwen3.5-4B and Qwen3.5-9B; the model’s four-loop Humanity’s Last Exam score is 10.43, compared with 11.31 for Qwen3.5-4B. NVIDIA says it measured accuracy using a consistent evaluation harness, but flags its Artificial Analysis v4.3.2 estimates as internal results that may differ from official scores.
One cache option limits memory growth
Looping can raise memory use if the runtime keeps a separate key-value cache for each pass through the shared block. Dory’s model card describes a last-loop caching mode that retains the cache for the final pass instead, keeping the cache footprint at the one-loop size regardless of loop count. That mode trades stored intermediate loop information for lower cache use, which can leave more memory available for longer contexts or concurrent requests.
The release comes in BF16, MXFP8, NVFP4 and Q5_K_M GGUF formats. The GGUF download is a quantized export sized at about 3.67 GB. NVIDIA’s setup guide says the benchmark results are for BF16, not the downloadable GGUF file. Its setup guide also says llama.cpp support is still awaiting inclusion in a release, and directs users to a Dory-enabled build in the meantime.
The next test is workload-level performance
Dory’s main proposition is configurability: the same 4B checkpoint can spend different amounts of compute on a task, with the setting changed at inference rather than through retraining or a model swap. The published results support a narrower claim than a general performance win: more loops improve some coding scores, while other measures flatten or decline, and the model card’s accuracy comparisons are NVIDIA’s own evaluations.
For users choosing among loop counts, NVIDIA’s guidance is specific: start at two, then compare latency and cost on the target workload before moving to one or four. The Q5_K_M GGUF file listed on the card is 3,673,288,864 bytes.