The Pulse
Cactus Ships Needle 3, a 9–29MB On-Device Tool-Calling Model
Cactus Compute has released Needle 3, a compact model for local tool selection, structured extraction and text embeddings on phones, wearables, robots and other small devices. The product page reports 9–29 MB CQ2 binaries and performance re
AI.info Team ·
Cactus Compute is betting that many device assistants do not need a general chatbot. The company has released Needle 3, a model built to select tools, fill arguments and return structured data locally, with shipped binaries ranging from 9MB to 29MB. Cactus says the model gives up general chat capacity to compete with models up to 10 times larger on mobile tool-calling tests.
The release puts Cactus in a direct disagreement with the usual design for device agents: send a request to a large cloud model and wait for it to decide what action to take. Needle 3 instead keeps the model, tool schemas and inference engine on the device. Its intended targets include phones, watches, augmented-reality glasses, cars, robots, smart-home equipment and microcontroller-class boards.
Needle 3 narrows the job to actions and data
Needle 3 handles three related tasks. It chooses among functions exposed by an application and extracts the arguments needed to call them; it maps unstructured text into a developer-defined schema; and it produces text embeddings for local search, matching and routing.
That narrow scope changes what the model needs to do. An assistant asked to dim a room and lock a door should return two calls in sequence. A request for an unsupported action should produce an empty list rather than a plausible-looking guess. For extraction, Cactus says the decode grammar guarantees that output conforms to the requested structure.
The model also reports a confidence score with each response. Cactus's runtime can withhold a call below a confidence floor, allowing an application to act automatically on high-confidence results, ask for confirmation on uncertain ones or refuse when no suitable tool is found. The release page gives an example response containing a light-control call, a reasoning field, a confidence score of 0.94, throughput measurements and peak memory use.
The 9–29MB range comes from one model ladder
Needle 3 is based on what Cactus calls a Laddered Simple Attention Network. Developers can select a subnetwork from two to 20 layers, with each depth treated as a deployable model. Cactus says the 20-layer version contains 121 million parameters but performs the arithmetic of a smaller model because many parameters sit in its engram memory component.
The release uses CQ2 quantization, which Cactus identifies as a 2-bit weight format. The product page says the shipped CQ2 binaries range from 9MB to 29MB. It does not provide a per-depth size table. The same engine can load each depth, letting developers trade accuracy, memory and speed for the device class they are targeting.
On a Raspberry Pi 5, the company reports decode speeds ranging from roughly 400 tokens per second at the top of the ladder to 4,000 tokens per second at the bottom. Prefill performance ranges from 1,000 to 10,000 tokens per second. Those figures are Cactus's measurements, not an independent hardware evaluation.
Cactus claims an advantage on mobile tool calls
Cactus evaluates Needle 3 on three tool-calling suites: Mobile Actions, DroidCall and BFCL v4. The first covers phone commands mapped to Android intents across 961 rows; DroidCall contains 200 phone requests, including cases requiring multiple calls in order; and BFCL v4 measures single-turn function calling across 3,641 rows.
For extraction, the company reports results on DSTC8 dialogue slots and the SNIPS dataset, including both field-level extraction and seven-way schema selection. Cactus says Needle 3 beats models 10 times its size on mobile tool calls and matches models two to three times larger on extraction. The comparison includes larger models running with higher-precision weights under vLLM, as well as DeepSeek V4 Flash accessed through its cloud API.
Those comparisons describe a specialized contest rather than a general language-model ranking. Needle 3 is not presented as a replacement for a model that writes long documents, answers broad factual questions or conducts open-ended conversations. Its design assumes that the application already knows which functions or schemas are available and needs a compact component to route requests into them.
Deployment targets phones and small boards
Cactus is shipping platform-specific engines alongside the model. The release lists targets for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, WebAssembly and WASI. Each engine is under 1MB and loads the model weights separately, allowing the same weights to be paired with different native libraries or browser runtimes.
The Python package can fetch the engine and cache it locally, while the command-line workflow can build a deployment for a selected platform and layer depth. The page shows Python workflows for tool calling, extraction and fine-tuning, along with commands for building a deployment for a chosen platform and subnetwork.
Fine-tuning is where the smallest versions are meant to work
Cactus positions the smaller subnetworks as starting points for product-specific fine-tuning rather than universal assistants. The company says fine-tuning on its DroidCall task improves every subnetwork by 18 to 36 points, and that tuned versions from four layers upward pass DeepSeek V4 Flash on the downstream comparison it publishes.
The workflow keeps the full 20-layer model as the training base, applies LoRA to the frozen weights and then exports a four-bit .cact file for the chosen device. A developer can therefore train against the full model, then produce a two-, four-, eight- or 20-layer deployment without changing the runtime interface.
The practical claim is narrow but measurable: tool selection, structured extraction and local routing in a package small enough to sit beside the application it controls.