Skip to content
AI.info

The Pulse

AMD says Unsloth brings NPU inference to 100-plus agentic AI apps and workflows

Unsloth Desktop now connects compatible applications to models running on AMD XDNA 2 neural processing units in supported Ryzen AI PCs. AMD says the integration reaches more than 100 agentic AI applications and workflows.

AMD says Unsloth brings NPU inference to 100-plus agentic AI apps and workflows

Unsloth Desktop users can now run supported AI models locally on AMD XDNA 2 neural processing units in compatible Ryzen AI PCs. AMD says the integration connects that NPU inference to more than 100 compatible agentic AI applications and workflows.

Source newsroom.amd.com

AMD XDNA 2 is the first NPU architecture supported by Unsloth Desktop, according to the company’s November 19 announcement. The integration gives supported model inference an alternative to GPU execution, leaving CPU and GPU resources available for other applications.

How requests reach the NPU

A compatible application sends a model request to the Unsloth Desktop API. Lemonade and FastFlowLM handle model serving and inference on the NPU, and the response returns to the application for use in its workflow.

Unsloth Desktop offers an OpenAI-compliant endpoint for compatible applications. The tools named by AMD include Hermes Agent, OpenClaw, Open WebUI, Cline, LangChain and GitHub Copilot.

Unsloth also provides open-source tools for fine-tuning models and a desktop application for training and inference without writing code. The NPU integration extends the desktop ecosystem beyond CPU and GPU execution.

Supported machines and setup

Unsloth lists Ryzen AI 300, Ryzen AI Max and Ryzen AI 400 processors with an XDNA 2 or newer NPU among supported hardware. Supported operating systems are Windows 11 and Linux on x64 systems.

On Linux, the documentation specifies kernel 7.0 or newer with the amdxdna driver, or an older kernel with amdxdna-dkms. It also calls for updated NPU firmware and an unlimited memlock setting.

NPU setup in the model picker downloads FastFlowLM and checks the NPU driver version. Once setup is complete, users can select an NPU model to download and load.

Models and controls

The NPU runs Q4NX 4-bit models built for the FastFlowLM runtime. Unsloth says supported GGUF models can be converted to Q4NX, although directly loading users’ own conversions from the NPU model list is not yet supported.

Its documentation lists features including reasoning, vision, tools, web search, chat with local documents and coding with local files. Compatible applications and scripts can connect to NPU models through Unsloth’s API endpoint.

Settings include context length and power modes named Power saver, Balanced, Performance and Turbo. Changes to those settings take effect the next time the model loads.

Local runtime

Unsloth manages separate copies of Lemonade and FastFlowLM, with their own runtime, configuration and model folders. Runtime downloads are pinned to specific versions and verified with SHA-256 checksums.

The managed Lemonade server accepts connections only from the user’s PC and uses a new authentication key each time it starts. Quitting Unsloth stops the managed runtime.

Sources

  1. AMD Newsroom
  2. Unsloth Documentation

Companies in this story

Related stories

Explore

More articles