Muse Glimmer 30B: An On-Device Agentic Model

AI summary

Overview

Research area: Natural Language Processing — compact open-weight language models for autonomous, tool-using agents running on consumer hardware.

Technical level: Intermediate. The architecture, distillation, quantization, and evaluation sections are approachable with basic familiarity with transformer language models; the mixed-precision quantization search (Greedy-KL, EvoPress, NVFP4/MXFP8) and the DFlash drafter are closer to Advanced.

Scope: This paper describes Muse Glimmer, a ~29.6B-parameter dense, text-and-image-input causal language model distilled from Meta's "Muse Spark" teacher, quantized to roughly 4-bit precision, and evaluated against Gemma4-31B and Qwen3.6-27B on agentic, tool-use, reasoning, multimodal, long-context, multilingual, and privacy/safety benchmarks.

What This Paper Is About

Publicly released models that fit on consumer hardware are typically trained as general-purpose chat systems and are not specialized for reliable autonomous agent behavior — sustained multi-step planning, precise tool invocation, and recovery from failed steps. The authors' goal is to close the gap between cloud-hosted proprietary agents and what a developer can run locally by building a model, Muse Glimmer, that is purpose-built for agentic tasks and fits inside a 32 GB dedicated-memory envelope without cloud infrastructure.

Key Contributions

1. A 30B agentic model for local deployment. Muse Glimmer is released as a ~29.6B-parameter dense model with a perception encoder, accepting text and image inputs and producing text outputs, trained on data from over a hundred languages, and quantized to approximately 4-bit precision for single-GPU inference on devices with 32 GB of dedicated memory. The authors report state-of-the-art results for its size class against Gemma4-31B and Qwen3.6-27B on tool use, multi-step planning, and agentic task completion, and release weights under the Apache 2.0 license.

2. Targeted distillation from a frontier teacher. The model is distilled from Muse Spark using co-distillation on the top 256 teacher logits, followed by supervised fine-tuning, on-policy distillation (student-generated rollouts scored per token by the teacher), and multi-objective reinforcement learning. The authors argue this yields a capable small model with lower inference compute cost.

3. A dense architecture chosen over Mixture-of-Experts under a fixed memory budget. The team trained architectural "ladders" comparing a dense baseline against MoE designs with 1:8 and 1:16 active-expert sparsity at a constant inference memory footprint, and found dense configurations gave a 2x–2.5x compute-efficiency advantage.

4. A deployment stack for consumer memory limits. The paper reports a mixed-precision quantization study (including a 19.65 GB EvoPress profile and a smaller Q4KM profile) plus a DFlash block-diffusion speculative-decoding drafter, and releases full-precision weights, 4-bit quantized weights, and the DFlash head.

Main Findings

• Agentic benchmark leadership, with exceptions. Muse Glimmer ("high" reasoning) scores 23.5 on τ³-Banking (Gemma4-31B 15.1, Qwen3.6-27B 16.7), 47.6 on WildClawBench (37.6 / 43.2), 43.3 on GAIA2 (36.4 / 40.0), 75.5 on MCPAtlas (54.2 / 62.5), 51.2 on SWE-Bench Pro (36.9 / 50.2), and 76.0 on SWE-Bench Verified (66.6 / 77.2). It does not lead every row: Qwen3.6-27B is higher on GDPval-AA (1141 vs 953), OSWorld-Verified (75.6 vs 65.9), TerminalBench 2.1 (60.7 vs 51.7), and SkillsBench (46.6 vs 44.3).

• Reasoning and knowledge. AIME 2026 is 94.7 (Gemma 89.2, Qwen 94.1), GPQA Diamond 83.5 (85.7 / 84.2), HLE 22.0 (23.6 / 23.1), DeepSearchQA 74.6 (61.7 / 71.1), IFBench 77.0 (76.0 / 70.8).

• Multimodal scores are close across the three models. CharXiv Reasoning 78.8 (77.7 / 78.4), MMMU-Pro 74 (73 / 75), OmniDocBench v1.5 75.8 (72.5 / 77.8), Screenspot Pro 75.4 (75.9 / 76.1).

• Long context and multilingual gains are the largest margins. Beam 128K 65.1 (58.2 / 63.0), AA-LCR 80.0 (68.3 / 73.3), Omnilingual Gaia2 60.4 (37.5 / 45.0), Linguini 8.8 (3.8 / 2.5).

• Privacy and safety results are mixed. On CIMemories, violation rate (lower is better) is 26.4 versus Gemma4-31B's 12.1 and Qwen3.6-27B's 53.4, while coverage is 64.8 (53.0 / 66.9). On Siren AgentDojo, attack success rate at 5 attempts (lower is better) is 28.4 (25.6 / 40.3) with utility 94.2 (90.8 / 92.7) — so Gemma4-31B resists injection better, but Muse Glimmer keeps higher task utility.

• Dense beats MoE at equal memory. Ladders used 550 target tokens-per-parameter for dense and 2,800 (1:8) and 6,000 (1:16) TPP for the MoE variants; the dense curve showed a 2x–2.5x compute-efficiency gain, motivating the dense design.

• MLA was tested and rejected. A simplified Multi-Head Latent Attention variant in global layers, configured either to match the baseline head count or its total KV footprint, underperformed GQA on long-context benchmarks such as RULER.

• Quantization is nearly lossless. The BF16 model occupies 55.7 GB; over 15 downstream benchmarks, average accuracy is 75.7 for BF16 versus 75.6 (EvoPress), 75.3 (Greedy-KL), 74.8 (Q4KM), and 74.4 (NVFP4/MXFP8). Average response lengths were 11070, 11417, 10754, 10774, and 10988 tokens respectively — the authors report no significant change in chain-of-thought length relative to BF16.

• Deployment profiles are selected by memory tier. EvoPress gives the lowest validation loss and is used for 32 GB deployments (19.65 GB of weights; 51/130/237 weight units at Q4_K/Q5_K/Q6_K, group size 32), while Q4KM gives the smallest footprint and is used for 24 GB deployments.

Methodology in Plain English

The team started from a memory constraint rather than a capability target: the model must run inside 32 GB of VRAM. They trained several scaled-down model families — one dense, two mixture-of-experts — at the same inference memory footprint and compared how quickly quality improved per unit of training compute; the dense family won, which set the roughly 30B dense design. The backbone is 52 transformer layers with grouped-query attention (32 query heads, only 2 key-value heads) and a repeating pattern of three local-attention layers (2048-token window) followed by one global layer, which keeps the KV cache from growing without bound during long agent sessions. Images are handled by a frozen ~1.8B-parameter vision encoder whose output tokens are compressed 2x2 and projected into the language model's embedding space.

Training is a distillation pipeline. The model learns from the top 256 logits of a much larger in-house teacher (Muse Spark) for nearly all of pre-training, then switches to ordinary next-token prediction for the final 5% to correct for the distortion introduced by truncating the teacher's distribution. Checkpoints from the end of pre-training are averaged together. Two mid-training phases then add domain-specific and agentic data and stretch the context window from 32K to 128K. Post-training adds supervised fine-tuning on synthetic and human data, on-policy distillation (the student generates its own trajectories and the teacher grades each token), and reinforcement learning against reward models and task-success verifiers.

Much of the training data is synthetic: personas are generated from demographic and personality attributes, placed in simulated "universes" (email, calendar, documents, WhatsApp, code repositories, command-line tools) with 10–14 contacts and weeks of correspondence, and used to produce multi-step tasks with verifiable rubrics. Trajectories that fail, hallucinate, or contain undesirable behavior are filtered out before training.

Finally, the model is compressed for shipping: mixed-precision quantization schemes are compared by their change in negative log-likelihood versus BF16 and then checked on downstream tasks, and a small block-diffusion "drafter" model is trained to propose token blocks for speculative decoding.

Why This Matters

Impact on research. The paper is a concrete data point that distillation from a frontier teacher plus careful systems work can produce a small model that is competitive on agentic tasks — and that architecture choice should be driven by a deployment memory budget rather than by parameter count alone. The dense-versus-MoE comparison at fixed memory and the negative result on MLA are useful reference points for other small-model efforts.

Real-world applications:

• Local coding agents: SWE-Bench Verified 76.0 and SWE-Bench Pro 51.2, plus TerminalBench 2.1 at 51.7, indicate viability for repo-level and shell-based development assistance without sending code to a server.
• Personal assistants over private data: the email/calendar/messaging agent scenarios and the CIMemories privacy evaluations target assistants that must decide what information is appropriate to share on a user's behalf.
• Document and screen understanding on device: OmniDocBench v1.5 75.8, CharXiv Reasoning 78.8, and Screenspot Pro 75.4 support reading charts, scanned documents, and UI screenshots locally.
• Multilingual local agents: Omnilingual Gaia2 at 60.4 and Linguini at 8.8 point to agent behavior in languages beyond English, relevant for markets where cloud access is limited or data must stay local.

Industry relevance. An Apache 2.0 release of full-precision weights, 4-bit weights, and a speculative-decoding head plugs directly into the existing local-inference stack (the paper cites llama.cpp, MLX, and ExecuTorch by name, and modifies the Q4_K_M and Unsloth Dynamic-style recipes). That makes the model usable not just by researchers but by developers shipping offline, latency-sensitive, or privacy-constrained products on 24 GB and 32 GB hardware.

Future Directions

• Video understanding. The paper states the model can ingest video as a sequence of frames but is not explicitly optimized for it, leaving frame-efficient video training as an open direction.
• Closing the remaining gaps. Muse Glimmer trails Qwen3.6-27B on several rows (GDPval-AA, OSWorld-Verified, TerminalBench 2.1, SkillsBench, OmniDocBench), and trails Gemma4-31B on CIMemories violation rate and Siren AgentDojo attack success — targeted work on computer-use and injection robustness is a natural next step.
• KV-cache and memory efficiency. Since the authors document that long agent sessions are dominated by KV-cache size, and their MLA attempt underperformed GQA on RULER, better cache-compression schemes that hold up on long-context benchmarks remain an open problem.
• Missing deployment measurements. The paper does not report tokens-per-second, latency, or DFlash acceptance rates for the chosen quantization profiles, so the actual speed benefit of speculative decoding on edge hardware is not quantified in the content provided. The provided text also ends partway through the related-work section, so the authors' own stated limitations and future work are not reported.

Target Audience

Engineers and researchers building on-device or privacy-preserving AI agents; practitioners who need to choose between dense and MoE architectures under a fixed memory budget; quantization and inference-optimization specialists interested in mixed-precision allocation and speculative decoding for small batch sizes; and product teams evaluating whether an open-weight 30B model can replace a cloud API for agentic, multimodal, or multilingual workloads. Readers looking for a formal scaling-law study or a detailed training-compute accounting will not find it here.

Authors’ abstract

We introduce Muse Glimmer, an open-source 30B parameter causal language model with perception encoder, purpose-built for autonomous agentic tasks on consumer hardware. Muse Glimmer integrates multi-step reasoning, reliable tool use and failure recovery into a single model that runs locally on consumer devices without requiring cloud infrastructure. The model accepts text and image inputs and produces text outputs, was trained on data from over a hundred languages, and is quantized to approximately 4-bit precision for efficient single-GPU inference. Muse Glimmer achieves state-of-the-art performance for its size class on agentic benchmarks, setting a new standard compared to the models of its size: Gemma4-31B and Qwen3.6-27B on tool use, multi-step planning, and agentic task completion. We release the model under the Apache 2.0 license to foster open-source development of capable, safe, local AI agents.