nanoMuse: An Open-Source Personal Agent for Every Device You Own

AI summary

Overview

Research area: Personal AI agents — system design, open-source deployment, safety architecture, and agent governance.

Technical level: Intermediate. The paper assumes familiarity with tool-calling agents, screen-operating models, and cloud-vs-device deployment, but it presents no new algorithms and reports no benchmark results.

Scope: A position-and-system report that defines the "personal agent," reconstructs Meta's Muse from public documents and a copy of its production prompt, and presents nanoMuse — an open-source (GPL-3.0) personal agent that runs on a person's own devices — as the open counterpart.

What This Paper Is About

Assistants since 2011 answered questions and waited; agents from 2023 did a task and stopped. In September 2026 Meta released Muse, an agent for one person with accounts, devices, lasting memory and a conversation that persists — but closed, running in Meta's cloud in one Linux VM per person, sold in the United States only, with only a small gadget SDK (October 2, Apache licence) open. The paper's goal is to argue that an agent holding a person's life should be inspectable and runnable by that person, and to present nanoMuse as a complete open alternative: one agent on every device the person owns, with hands on the phone's screen and the computer's, sharing one conversation over an open relay, gated by a Sentinel, storing memory as files the person can read, and using a model of the person's choosing.

Key Contributions

1. A definition of the personal agent — a program that acts for one person on their accounts, devices and files, over weeks, and answers afterwards with a log, a ledger and a list of permissions — framed as five questions (what it knows, where it acts, whom it answers to, how it learns, how it is paid for and governed) and three horizons.
2. A sourced reading of how Muse is built, from four public Meta documents plus a copy of Muse's production system prompt, with every statement marked as documented, prompt, observed, or inferred.
3. nanoMuse itself, described as of version 0.1.40 (October 2026), with its device architecture, Sentinel decision order, screen-operating "Hands," file-based memory, relay data policy, provider catalogue, and size/cost estimates.
4. A community roadmap and an explicit limitations section, listing what nanoMuse does not yet handle and what the authors would not yet trust it with.

Main Findings

• Muse is a per-person cloud computer. Each person gets a dedicated Linux VM in Meta's cloud as the system of record for files, memory, credentials and conversation; the agent and its tools run in a container with a seccomp filter and reduced capabilities. The prompt copy indicates deployments are replaced about hourly while the home directory persists, with a real Chromium keeping cookies and tabs.
• Muse's safety mechanisms sit outside the agent's reach. The Sentinel is described as "the sole permission authority" for connector actions and all network egress; where a secret is needed the agent holds only a surrogate token; tool processes are tracked in the kernel and become "tainted" on reading user data; approvals are scoped capabilities (one-time, per session, per task, time-bounded or standing).
• Muse's browser safety trades capability for safety. The browser sub-agent sees an accessibility-tree snapshot rather than the DOM and is paused while a credential the model never sees is entered; a checkout detector forces approval with the exact amount. The authors infer this means Muse cannot operate an interface with no accessibility tree — a native phone app, a kiosk, a game.
• The field converged on one architecture in five weeks. Muse (Sep 8, 2026), Today (Sep 15), CopilotKit's OpenMuse (Sep 15), Manus Cue (Sep 28) and OpenAI's "dots" (Sep 29) all place a computer per person or per agent in a vendor's cloud with thin clients on the person's devices and a vendor-trained model.
• nanoMuse puts the agent on the device instead. The Android app is a modified OpenMinis (a complete agent inside the app with a sandboxed Alpine Linux, shell, browser, MCP servers, skills and scheduled tasks); the desktop app is an Electron shell around the DeepSeek harness with a nanoMuse plugin and an embedded Python runtime; the relay is one Python process and one SQLite file.
• The Sentinel is a policy gate with a fixed decision order. Denied tools; the person's rules; always-allow and always-ask lists; taint (once a conversation has read private data, any outbound call outside the allow-list becomes an ask, and never the reverse); tool risk versus the person's mode; then warnings no mode can wave through. Approvals are scoped grants — once, this conversation, and where a target can be named, always for that recipient, site, folder, device or application — listed under Permissions with a revoke button.
• Hands follow a ladder and are off by default. The agent tries a skill, a command-line tool or an MCP server; a page fetched with the person's login; the browser; and last the device's own screen — a one-action-per-screenshot loop in the dialect of MemGUI-Bench's phone operator, with the computer side ported from UI-TARS-desktop. The model outputs a thought, a sentence in the person's language saying what it is about to do, and one gesture; labels with committing words (confirm payment, transfer, place order, send, delete) make a step sensitive and approved every time.
• Memory is plain Markdown on the phone. SOUL.md (who the agent is), USER.md (who the person is), GLOBAL.md (what holds across conversations), a dated diary, and HEARTBEAT.md (routines). On the computer, each memory is a line with a change log and an undo, recalled by rare words or, with an embedding endpoint, by meaning.
• The relay keeps a stated minimum. The account (hashed identifier, issued keys, device each went to), the ledger of model calls (model, token counts, cost; never content), the agent's profile, hub presence, and the text of conversations the person chose to sync; never files, images or what a tool returned. Revoked-key hashes are kept for ninety days.
• No performance numbers are reported. The paper states there is no success rate and no count of human take-overs for the hands, and does not estimate one. Sizes and costs (Table 2) are given as measurements of release files and estimates from documentation and provider price lists.

Methodology in Plain English

The paper is a design and analysis report rather than an experimental study. The authors first reconstruct Muse from Meta's public record — the launch announcement, a safety-architecture essay, a product design essay, and a help-centre privacy page — plus a copy of the production system prompt whose provenance they say they cannot verify beyond its consistency with the other sources; each claim is labelled by source type. They then propose a normative framework for what a personal agent should be, expressed as five questions and three horizons, and measure their own system against it. Finally they document nanoMuse by construction: which upstream projects each device app is built from, how the Sentinel orders its decisions, how the on-screen loop is structured, what the memory files are named, what the relay stores, and how much the software weighs and costs. No controlled comparisons, benchmark runs or user studies are conducted.

Why This Matters

The paper argues that who runs a personal agent, where, and under whose eyes is not a deployment detail but the product — and that the open counterpart to Muse has to exist as a complete, runnable thing rather than as an SDK.

Real-world applications:

• Running a personal agent on hardware a person already owns, with an API key of their own and no account or server, so nothing leaves the device except model calls.
• Operating everyday apps that expose neither an API nor a usable web page — banking, government services — through the phone's own screen, which the authors identify as the gap cloud-computer agents cannot close.
• Self-hosting the device-meeting point (the relay) in Docker on the smallest server tier for households or small groups, with everyone billed to the owner's provider key.
• Keeping long-term memory as readable files a person can open, correct, delete and export, so the agent can outlive its models and devices.

Industry relevance: The paper is a direct challenge to the vendor-cloud model that Muse, Today, CopilotKit's OpenMuse, Manus Cue and OpenAI's dots share. It positions open code, device-local execution, choice of model provider (a catalogue of eighteen Chinese and international providers) and user-held memory as the alternative shape for the category.

Future Directions

• An evaluation suite for the hands. Built from failed runs so each release has a number rather than an impression, run beside AndroidWorld for the phone, OSWorld for the desktop, MemGUI-Bench for what the hands remember across apps, and OS-Harm for what they should refuse.
• A small open model for the hands, trained on traces people chose to contribute — the way MobileForge adapts an open model to real apps from runs it collected itself — so the most sensitive loop does not depend on a vendor.
• The physical world as hands. The hub's frames already carry look, act and ask, so a camera, appliance or robot arm would be one more device under the same approvals and the same log.
• Memory and identity that last years, surviving changes of model, device and vendor — plus the open problem of memory provenance, since the store currently records when a line changed but not which model wrote it, the date, or a confidence.

Target Audience

Agent and HCI researchers studying personal-agent architecture and governance; open-source and mobile developers who would build, self-host or contribute to nanoMuse; safety and privacy researchers interested in how permission boundaries differ between a policy gate in the same trust domain and a privileged host outside the agent's reach; and product and policy readers who want a sourced account of how Meta's Muse is constructed and a concrete alternative design.

Authors’ abstract

Assistants from 2011 answered and waited, and agents from 2023 did a task and stopped. In September 2026 Meta's Muse showed an agent for one person, with accounts, devices, memory and a conversation that lasts, closed, in a vendor's cloud, in one country. Such an agent is expected to act on a person's accounts and devices, remember them across weeks, speak first when it is worth it, and answer for what it did. It is a kind of software, not a model, and until now had no open counterpart. This report defines the personal agent in five questions and three horizons. It reads how Muse is built from Meta's public record and a copy of its production prompt, each statement marked by its source. It then presents nanoMuse, the open-source counterpart under the GPL-3.0, one agent on every device a person owns, with hands on the phone's screen and the computer's. They share one conversation over a relay anyone can run; every action goes through a Sentinel, memory is files the person can read, and the model is their choice. Its size and cost are given as estimates. What is open, memory with provenance, an evaluation suite for the hands and an open model for them, is set out as a roadmap.