Skip to content
AI.info

The Pulse

Microsoft publishes quantized Aion model for WebGPU

Microsoft has published a WebGPU version of its Aion Instruct language model with approximately 2.29 billion base-model parameters. The roughly 1.3 GB package can run through ONNX Runtime in browsers and native environments.

Microsoft publishes quantized Aion model for WebGPU

Developers can now download a roughly 1.3 GB WebGPU version of Microsoft’s Aion Instruct language model, quantized from a base model with approximately 2.29 billion parameters. The package targets graphics processors in browsers and native environments, according to the model card published by Microsoft’s Windows Applied Sciences Group on Hugging Face.

The model card was posted on November 18. The release uses the ONNX format and carries an MIT license.

Inside the package

The WebGPU version uses INT4 weights produced from a quantization-aware-trained checkpoint, with FP16 activations and key-value cache. It has a 32,768-token context length and lists English, German, Spanish, Portuguese, Italian, French, Chinese and Japanese as its languages.

The model is packaged as a single ONNX decoder graph using opset 21 and Microsoft’s contributed operations. Its attention architecture combines 21 sliding-window layers, each with a window of 2,048, and seven full-attention layers.

The linear layers use 4-bit symmetric block-wise weights with a block size of 32. The token embedding and tied language-model head also use 4-bit block-wise weights, with a block size of 64.

Running the model

The accompanying example uses ONNX Runtime GenAI with the WebGPU execution provider. The card says the model runs on ONNX Runtime builds that provide that execution provider, including ONNX Runtime Web in a browser.

Its maximum length for the prompt and generated tokens defaults to 4,096 in the supplied configuration. That limit can be raised to the model’s 32,768-token context, though the card notes that doing so requires a larger pre-allocated key-value cache. The example can also be run on a CPU by omitting the calls that select WebGPU.

The download includes the decoder graph, approximately 1.3 GB of external quantized weights, configuration files, a tokenizer and a chat template. The quantized model uses the same chat template as its base model.

Uses and limitations

The base model was primarily trained for general-purpose text tasks, including document comprehension, content analysis, question answering and natural-language understanding. Its card also lists summarization, rewriting and chat as tasks.

Microsoft says quantization can affect output quality and that quality relative to the bfloat16 base model will be reported on the WebGPU model card. In the base model’s card, the company cautions that the smaller model can miss details, make misstatements or fail at complex multistep reasoning. Output quality may also differ by language, and the model may be susceptible to adversarial attacks.

Windows transition

A further optimized version of Aion Instruct, paired with Microsoft’s content moderation model, is planned to become the Windows operating system’s small language model by the end of January 2027. The Hugging Face base-model card describes that release as distinct from the version published there.

In a separate Windows developer post, Microsoft said it planned to begin rolling out Aion Instruct to Windows Insider Preview devices on December 4, followed by a retail-device rollout in January 2027 to complete the transition from Phi Silica. The post says apps using highly customized prompts, custom LoRAs or, because of a new watermarking implementation, custom model output parsing will require action during the transition.

Sources

  1. Original source Microsoft · Aion 1.0 Instruct 2.3B ONNX WebGPU huggingface.co

Other pages read

  1. Microsoft · Aion 1.0 Instruct 2.3B huggingface.co
  2. Microsoft · Aion instruct local AI windows GPU devblogs.microsoft.com

Companies in this story

Related stories

Explore

More articles