Agents in the Large: Perception-Centered Architecture for Persistent Agents

AI summary

Overview

Research area: Natural Language Processing / language-agent architecture; specifically the conceptual design of long-lived, persistent AI agents.

Technical level: Intermediate. The paper is a conceptual and architectural framework rather than an empirical study, so it requires no specialized statistical background, but familiarity with existing language-agent designs (memory, tool use, reasoning loops) helps.

Scope in one sentence: The paper proposes Pera, a perception-centered conceptual architecture that describes how language agents should be organized to provide persistent service in long-lived settings rather than only completing bounded, user-specified tasks.

What This Paper Is About

Most language agents today are built as "agents in the small": systems of a language model, memory, tools, and a decision procedure organized around a single bounded objective given by a user. The authors argue that this framing is insufficient for agents meant to assist users continuously over long periods, where needs, context, and service procedures persist and change across many tasks.

The paper's goal is to supply the missing architectural account: which components must exist beyond the cognitive core, how an agent should sense changes in itself and its environment, and how those perceived changes become work the agent performs on itself. The authors name this framework Pera (Perception-Centered Architecture for Persistent Agents), drawing an analogy to software engineering's historical shift from "programming in the small" to "programming in the large."

Key Contributions

1. The Pera framework. A conceptual architecture for persistent agents that couples a lifecycle perception component with a control plane inside a "persistent agent core," layered above the conventional task-oriented cognitive agent.

2. A distinction between episodic and lifecycle tasks. Episodic tasks pursue a bounded user-facing outcome; lifecycle tasks are bounded tasks whose purpose is to maintain or improve conditions shared by future tasks (for example, updating stale context or revising an unreliable procedure). The paper states task type is determined by purpose and completion condition, not by whether the user or the agent initiates it.

3. A distinction between long-lived settings and long-horizon tasks. Long-horizon describes the extent of execution within one task; long-lived describes the scope of coordination across tasks, spanning multiple objectives and persisting after any one completes.

4. A retrospective organization of prior work plus a case study and forward-looking insights. The paper claims to use Pera to organize recent work, examine a detailed case study, and offer actionable directions for building future persistent agents. (The case study in Section 5 and the insights in Section 6 fall beyond the truncated excerpt provided, so their specific contents are not reported here.)

Main Findings

• Active perception is the missing layer. The paper argues that within a bounded task, the objective determines what information matters, but across task boundaries relevant information can remain significant after that objective ends. If information enters only through an active task's observation loop, the authors describe this as an "observability gap" across task boundaries — a gap about what the architecture makes available for control, not about storage capacity.

• Perception is necessary but insufficient. The same evidence may indicate a local exception, a persistent change, or a condition outside the agent's authority. Perception makes conditions available for judgment; a separate level of control must decide whether and how to respond.

• Signals are distinguished from observations. An observation is information supplied to a task runtime during execution to inform its next decisions. A signal makes a change available to the persistent agent core as the starting point of lifecycle-level judgment. The same event can serve both roles — for example, a tool error supports recovery within a task while also providing evidence that a reusable procedure has become unreliable.

• Signals are classified as external or internal. External signals come from physical sources (cameras, microphones, sensors measuring conditions such as temperature and motion), human sources (feedback on prior work, new needs), and digital sources (file-system events, updated access policies). Internal signals come from a single task execution, reviews across multiple task executions and internal context memory, or from perception and control themselves.

• Signal aggregation matters. A single failed tool call does not establish a persistent problem, but repeated failures following an interface change provide evidence that a reusable procedure has become obsolete. Aggregation lets perception recognize conditions that are hard to identify from one event alone.

• Lifecycle task packages are the dispatch interface. A package specifies the task objective, a completion criterion (the evidence the runtime must produce before stopping), an optional procedure, context acquisition requirements, recovery mechanisms (for tasks that modify persistent components), and a review criterion stating what must hold before a proposed change is accepted. Review is described as extending beyond completion: it examines evidence the runtime was not required to produce, particularly whether the change preserves behavior the agent already supported.

• Existing agents are characterized as having limited operational horizons by cited prior work. The paper cites prior empirical evaluations (Starace et al., 2025; Kwa et al., 2026; Xu et al., 2026a) reporting that recent agent systems have been run continuously for tens of hours on a single complex task, while their effective task horizons remain much shorter and progress often plateaus during extended runs. These are cited external findings, not results produced by this paper.

• No quantitative benchmarks, datasets, or measured results are reported in the provided content. The available text is conceptual and expository.

Methodology in Plain English

The paper does not run experiments. It is an architectural and conceptual contribution built through several steps.

First, the authors review three prior ideas: the software-engineering distinction between programming in the small and programming in the large (DeRemer and Kron, 1976), which separates computation inside a unit from control over relations among units; cognitive architectures for language agents (Sumers et al., 2023), which organize memory, actions, and decision procedures for bounded tasks; and active perception (Bajcsy; Hayes-Roth), which holds that what an agent senses is guided by its current activities and information needs, as illustrated by Hayes-Roth's architecture for adaptive intelligent systems and the Guardian system.

Second, they apply the software-engineering criterion — architectural scale is determined by the scope of activity and state placed under control, not by the amount of computation inside a unit — to language agents. This yields the claim that an agent is "in the small" when the episodic task is the highest-level unit it explicitly organizes, even if that task runs for many hours or the agent retains long-term memory, and "in the large" only when persistent conditions and cross-task relations themselves become objects of control.

Third, they derive three architectural requirements — organize a scope that persists across tasks, make changes to that scope observable, and convert relevant changes into bounded tasks — and map each to a construct: the long-lived setting, active perception across task boundaries, and lifecycle tasks.

Fourth, they assemble these constructs into the Pera diagram and describe its components: lifecycle perception (sensors plus signal processors), a control plane that evaluates perceived changes and formulates lifecycle task packages, a task agent specification that encodes the reusable decision procedure, and a task runtime instantiated for each task. They then specify a shared action space of seven actions — sensing, dispatching, instantiation, reasoning, retrieval, updating, and grounding — noting that reasoning, retrieval, updating, and grounding build on existing cognitive-agent actions while sensing, dispatching, and instantiation are added for perception and lifecycle-level coordination.

Fifth, they use the framework retrospectively to situate existing systems, citing proactive agents (ProactiveAgent, ContextAgent, ProAgent, ProactiveVA), runtime monitors, and scheduling systems (AIOS, DynTaskMAS, Agent.xpu, Agent libOS) as partial implementations of specific Pera components.

Why This Matters

Impact on research. The paper reframes persistent agency as an architectural rather than a purely capability-driven problem, and offers vocabulary — episodic versus lifecycle tasks, long-lived settings versus long-horizon tasks, signals versus observations — for comparing systems that currently address perception, scheduling, and self-modification in isolation. It also names understudied problems, such as how sensing intensity and granularity should adapt as the environment and agent state change, and how accurately to interpret signals and identify their underlying causes.

Real-world applications (as described or implied in the paper):

• Computer-use agents that monitor a user's activity and changes in the computer environment, and infer needs not yet articulated rather than only executing assigned tasks.
• User-preference maintenance, where a correction to a preference is intended to govern future outputs (the paper's example: correcting a preference that should govern future reports, as opposed to producing one report).
• Procedure repair after environment change, illustrated by a computer-use agent that repeatedly fails to submit expense reports after a reporting website changes its interface, requiring the reusable submission procedure to be revised and validated.
• Proactive assistance from multimodal sensing, as in systems cited by the paper (ContextAgent, ProAgent) that use video and audio from wearable devices to infer user context and emerging needs.

Industry relevance. The framework speaks directly to deployment concerns: dispatching must decide among immediate dispatch, delayed execution, and waiting for resources, dependencies, or authorization; lifecycle tasks that modify reusable procedures or source code can degrade later service if the update fails, so packages include stable-version retention and rollback; and modifications may be restricted to specified components or require additional authorization before consequential actions. These are operational and governance questions for any organization running long-lived agents, not just research topics.

Future Directions

• Adaptive sensing policies. The paper states that how sensing should adapt as the environment and agent state change remains understudied in recent language agents, and that persistent agents need to expand or revise what they monitor, adjust sensing frequency and granularity, and determine when related events should be considered together.

• Robust signal interpretation. Accurately interpreting signals and identifying their underlying causes is identified as a core challenge for lifecycle perception, including suppressing irrelevant inputs and combining signals whose significance emerges only over time.

• Scheduling lifecycle work alongside episodic work. Dispatching must consider how lifecycle tasks that maintain or improve future service are scheduled alongside episodic tasks with immediate response requirements.

• Self-improving package construction. The paper notes that if lifecycle tasks repeatedly expose deficiencies in package construction, revising the package-formulation procedure can itself become a lifecycle task — raising the question of how such recursive self-modification is bounded and reviewed.

• Governance of consequential changes. Deciding when a proposed modification requires authorization, and how acceptance, further revision, or rollback is determined by the control plane, remains an open design question.

Target Audience

Researchers and architects working on language agents, agent memory, tool use, and long-running autonomous systems; engineers designing production agent platforms that need scheduling, authorization, and rollback; and readers interested in the conceptual transfer of software-architecture ideas (programming in the large, active perception, self-adaptive systems) to AI agents. The paper is most useful to those who want a design vocabulary and an organizing framework rather than experimental results or implementation recipes.

Authors’ abstract

Cognitive language agents have achieved substantial progress by equipping language models with memory, tools, and decision-making procedures, enabling agents to reason and act in interactive environments. Existing frameworks largely cast these agents as systems for solving user-specified, bounded tasks. An increasingly important goal is for language agents to provide persistent assistance in long-lived settings where user needs, context, and service procedures persist and change, and to remain useful across the broad range of tasks that arise over time. Yet we still lack a framework to characterize persistent AI agents, organize existing work, and guide future development. To this end, we propose a Perception-Centered Architecture for Persistent Agents (Pera). Pera describes a persistent agent organized around perception and control components that continually perceive service-relevant signals from episodic task executions, internal context, and changes in the surrounding environment, and use these signals to construct lifecycle tasks. These tasks drive the ongoing operation and adaptation of the agent's service procedures. We use Pera to retrospectively organize recent work, examine a detailed case study, and offer forward-looking insights for building more capable persistent agents. Just as software engineering moved from programming in the small to programming in the large, Pera frames the evolution of language agents as an analogous architectural transition toward long-lived, adaptive intelligence systems.