The Pulse
Microsoft Completes Pretraining of Its 2.8T-Parameter MAI-Base-2
Microsoft AI says it has completed pretraining MAI-Base-2, a mixture-of-experts model with 2.8 trillion total parameters and 80 billion active. The company says its training method used 4.2–12.3 times less compute than DeepSeek-V4-Pro-Base
AI.info Team · Written with AI ·
Microsoft AI says MAI-Base-2 outperforms every open-weight model it tested, but the company’s own comparison has a notable limit: several rival models were available only as post-trained checkpoints, which Microsoft says may affect the results. The company published its account of the model’s pretraining on October 15, describing a 2.8-trillion-parameter system that is still undergoing post-training.
“MAI-Base-2 outperforms every open-weight model we tested on our held-out evaluations.”
Microsoft AI
2.8 trillion parameters, 80 billion active
MAI-Base-2 is a mixture-of-experts model with 2.8 trillion parameters in total and 80 billion active, according to Microsoft. The company trained it across 16,000 GB200 GPUs for 38 days in August and September 2026. Microsoft reports an average model FLOPs utilization of 21% for the training run and a goodput of 89%—the share of time training made progress.
The model’s scale reflects a change in architecture as well as size. Microsoft says it adopted a mixture-of-experts design in every layer, after changes to its implementation raised the utilization of those layers from 24% to 35%. The company identifies dynamic routing and the overhead from adding more, smaller experts as two sources of inefficiency it addressed.
Microsoft’s efficiency claim depends on its test method
Microsoft’s detailed comparison estimates that its training recipe needs 4.2–12.3 times less pretraining compute than DeepSeek-V4-Pro-Base to reach the same performance on four held-out evaluations. The company also reports a 6.3–12.7-fold improvement in FLOPs efficiency over MAI-Base-1. Those ranges describe compute estimates from scaling-law fits, not a direct claim that MAI-Base-2 is that many times faster or better at every task.
The evaluations cover held-out text, code, STEM, math and question answering. Microsoft says it used bits-per-byte for held-out text, its own codebase for code tests, and vendor-commissioned sets for STEM, math and Q&A; it also says it checked the evaluations for contamination against training data. DeepSeek-V4-Pro-Base is the reference base model in the efficiency analysis. For Kimi K3, Qwen3.8-2.4T-A95B and MiMo-V2.6-Pro-RL, Microsoft says only post-trained checkpoints were available and warns that post-training may change results.
A 50-trillion-token mix includes vision
Microsoft says its training mixture contained 50 trillion tokens and that its corpus of unique training tokens grew by roughly 36% from MAI-Base-1. The company says it processed the training tokens in-house and used organic data rather than distillation from other models. Its stated knowledge cutoff for MAI-Base-2 is March 2026.
The model was pretrained with vision using what Microsoft describes as an encoder-free vision-language architecture. Its vision-language corpus contains 4.5 trillion unique tokens; with repetition, vision-language data accounted for 18% of the 50-trillion-token training mixture. Microsoft lists image captioning, chart understanding, visual grounding, spatial reasoning, medical understanding and computer use among the covered tasks.
Weights are planned for Foundry partners
MAI-Base-2 is not presented as a finished assistant. Microsoft says post-training is underway and that the model will form the foundation for its next generation of models. The company says it plans to license the weights to trusted partners through Foundry, but its October 15 post gives no release date or pricing.
Microsoft also says preparations for MAI-Base-3 are already underway. For now, the concrete milestone is narrower: MAI-Base-2 has completed pretraining, and the company has published its training setup and evaluation claims while the model moves through post-training.