The Pulse
PrismML Fits a 27B Multimodal Model Into 5.9GB
PrismML released Bonsai 2 27B, a ternary multimodal model based on Qwen3.8 27B. The company says the model retains 98.2% of the baseline’s aggregate benchmark performance while using a 5.9GB model footprint.
AI.info Team ·
PrismML says it has compressed a 27-billion-parameter multimodal model into a 5.9GB footprint, releasing Bonsai 2 27B on September 17 as an open-weight system for local use. The company reports that its ternary model retains 98.2% of the aggregate benchmark performance of the full-precision Qwen3.8 27B model it is based on.
The release targets a familiar hardware constraint: a 27B model normally requires far more memory than a typical laptop or mobile device can provide. PrismML’s design uses weights represented as −1, 0, or +1, combined with FP16 group-wise scaling, producing what the company describes as 1.76 effective bits per weight.
PrismML’s technical announcement presents the result as the second generation of its Bonsai family, following the company’s July release of Bonsai 27B. The new model keeps the same basic focus on local inference but raises the reported quality level against its full-precision reference.
5.9GB for a model built from 27 billion parameters
Bonsai 2 27B accepts both text and images, supports a 262,000-token context window, and is released under the Apache 2.0 license. PrismML says the low-bit representation runs across the language model rather than being limited to selected components.
The company’s benchmark table gives Bonsai 2 27B an overall score of 83.9, compared with 85.4 for full-precision Qwen3.8 27B and 83.6 for the earlier Qwen3.6 27B baseline. The results cover agentic tool use, coding, instruction following, knowledge and reasoning, mathematics, and vision.
Performance varies by category. PrismML reports a 96.57 score in its combined mathematics tests, compared with 97.06 for Qwen3.8 27B. Coding scores 81.58 against 82.17 for the full-precision model, while the vision category reaches 78.59 against 81.64.
Reasoning and vision move onto local hardware
PrismML says Bonsai 2 27B runs on NVIDIA GPUs through CUDA and on Apple devices through MLX, using custom low-bit kernels. The company reports throughput of up to 143 tokens per second on an NVIDIA GeForce RTX 5090 and 46.8 tokens per second on an M5 Max.
The release also includes claims about energy use. On an RTX 4090, PrismML reports consumption of 0.714 milliwatt-hours per token, which it says is 40% lower than an 8B model running in full precision. Those figures come from the company’s own testing and are dependent on hardware, software builds, prompt length, context settings, and generation mode.
PrismML positions the model for coding agents, computer-use systems, private document analysis, multimodal debugging, and tool-calling workflows. The company’s examples include coding-agent loops and computer-use demonstrations on an RTX 5090, but the announcement does not establish how Bonsai 2 compares with commercial systems in broad, independent testing.
The deployment details behind the headline number
The 5.9GB figure describes the model footprint cited by PrismML, not necessarily the total disk space required for a working installation. The company’s Bonsai demo repository says its default setup downloads a 7.8GB PQ2_0 model package plus a vision projector, with additional space used by optional components such as Open WebUI and a code interpreter.
The repository also carries an important compatibility condition. Bonsai 2 currently requires binaries from PrismML’s fork of llama.cpp because the model depends on a Hadamard activation transform that has not yet reached the upstream project. The documentation warns that stock llama.cpp builds may refuse some Bonsai 2 formats or produce invalid output for another transitional format.
PrismML provides GGUF and MLX versions through its Hugging Face organization. The repository describes Bonsai 2 as the default model in its setup scripts, with support for Mac, Linux, and Windows paths that use Metal, CUDA, Vulkan, ROCm, or CPU execution depending on the machine.
Compression claims still need independent testing
The reported 98.2% retention figure is an aggregate across the company’s selected benchmark suite. That makes it useful for comparing PrismML’s stated target with its reference model, but it does not mean every task loses only 1.8% of quality. The category-level scores show larger gaps in vision and smaller gaps in mathematics and coding.
The result is still significant for developers trying to keep inference close to the user. A model with image input, long context, tool calling, and multi-step reasoning can be difficult to place on consumer hardware when its weights occupy tens of gigabytes. A sub-6GB model changes the memory calculation, even if the runtime package and auxiliary files require more room.
Bonsai 2 27B is available now, but its practical reach depends on the custom runtime, the selected file format, and the hardware used to run it. PrismML’s published numbers establish the company’s compression and benchmark claims; broader comparisons will determine how well the model performs outside those controlled conditions.