The Pulse

DwarfStar Adds Qwen3.8 and DeepSeek V4.1 for Local Runs

DwarfStar, Salvatore Sanfilippo’s local inference engine, added DeepSeek V4.1 Flash and Qwen3.8 Flash Next in mid-September. Its guides show very different memory needs: Qwen3.8 fits a 64GB Mac by keeping its n-gram table on SSD; V4.1 needs

AI.info Team · Written with AI · 2026-10-03

DwarfStar’s push to run open-weight models on local machines comes with a qualification: some supported models need data streamed from an SSD rather than held entirely in memory. Support for DeepSeek V4.1 Flash landed in the project’s GitHub repository on September 12, 2026, for Apple’s Metal, with CUDA text inference following a day later; Qwen3.8 Flash Next, contributed through pull request #991, was merged on September 14. The same mid-September changes brought broader vision support and a new path for smaller machines.

DwarfStar is a model-specific inference engine, not a general loader for arbitrary model files. That narrower approach gives it room to tune individual model paths, but it also makes support dependent on tested model formats and hardware.

The update adds two model paths

The new support covers two different memory challenges. DeepSeek V4.1 Flash brings its own tokenizer, inference graph, model files and vision encoder; files built for the earlier V4 Flash cannot substitute. Qwen3.8 Flash Next creates a smaller entry point, with a Q2 configuration designed to run on Apple Silicon Macs with 64GB of unified memory. The project’s current repository also lists GLM models and DeepSeek V4 PRO among its supported targets.

DwarfStar’s design is deliberately selective: its engine expects project-produced GGUF files, whose tensor layouts and quantization choices match its model-specific code. The project’s README says its tests cover model loading, prompt rendering, tool calls, cache state, the server and the coding agent together. Salvatore Sanfilippo, the Redis creator known as antirez who started DwarfStar, set out the reason for the project in an earlier post on his blog, “A few words on DS4”:

“AI is too critical to be just a provided service.”Salvatore Sanfilippo (antirez), creator of DwarfStar

Qwen fits a 64GB Mac by putting data on disk

The Qwen3.8 Q2 download totals 137.10 GiB, but DwarfStar keeps 41.73 GiB of main-model and MTP weights in unified memory. Its 95.37 GiB BF16 n-gram table stays on a local SSD and is read as needed. The setup guide recommends starting with an 8,192-token context and a 1,024-token prefill chunk, leaving memory for the context and runtime buffers.

That arrangement does not make a 137 GiB file fit into 64GB of RAM. It is also not the same as DwarfStar’s SSD expert-streaming mode: the Qwen n-gram table is a disk-backed part of the model’s execution, while expert streaming brings selected routed weights into a bounded cache. The 64GB configuration is documented for Macs. The project’s Qwen guide says ROCm, tensor parallelism and SSD expert streaming are not supported for Qwen3.8 yet.

DeepSeek V4.1 pushes beyond one machine’s memory

V4.1 Flash illustrates the larger end of the tradeoff. Its Q2 package is 341 GiB, including 152 GiB of main weights and 189 GiB of Engram tables that remain on disk. On a single 128GB Mac or DGX Spark, the main weights use SSD streaming; two 128GB machines can split the Q2 model between them. The Q4 package grows to 483 GiB, with 294 GiB of main weights, and requires streaming or a 512GB Mac for full residency.

Support varies by model and backend. V4.1 vision works on Metal, while CUDA support is text-only, according to the project’s model guide. Streaming expands what a machine can load, but it does not remove the need to budget for context, scratch space and other runtime memory. DwarfStar’s guide warns that streaming trades speed for capacity.

Benchmarks show why the model matters

The recorded baseline in the project’s performance guide measures DeepSeek V4 Flash Q2, not the newly added paths. On an M5 Max with 128GB, its recorded Q2 run at a 32,768-token context reports 557.04 tokens per second for prompt prefill and 34.36 tokens per second for generation. A 128GB DGX Spark records 855.94 tokens per second for prefill and 14.43 for generation at the same context.

Those figures are recorded baselines, not fresh measurements of every software change, and prefill and generation are reported separately. The September changes add new model paths, while the hardware guides show that model choice, quantization, memory, context and SSD behavior determine what “local” means for each configuration.