From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents

AI summary

Overview

Research area: LLM-based planning for embodied multi-agent systems — specifically decentralized partially observable environments where agents must cooperate with limited perception and costly communication.

Technical level: Intermediate. The paper is readable without deep reinforcement-learning background, but it uses DEC-POMDP formalism, tree-structured search, and LLM prompting terminology.

Scope (one sentence): The paper proposes PCE (Planner–Composer–Evaluator), a framework that converts the implicit assumptions already present in LLM reasoning traces into a scored decision tree so embodied agents can act under uncertainty without heavy inter-agent communication.

What This Paper Is About

Embodied agents working in partially observable, decentralized settings never see the whole world, so they constantly face uncertainty about hidden objects and about what their partners intend to do. The standard fix — talking to each other constantly to verify plans and exchange information — burns tokens and time, and when a human is in the loop it can disrupt their workflow. This paper shows that LLM planners already generate useful assumptions about uncertain parts of the environment inside their own reasoning traces, and it proposes harvesting those assumptions into an explicit, scored decision tree so agents can choose actions rationally instead of communicating heavily.

Key Contributions

1. The PCE framework. A modular architecture that redesigns the planning module into three stages: a Planner that produces a reasoning trace and a candidate action, a Composer that extracts and structures the implicit assumptions from that trace into a decision tree (internal nodes = environment assumptions with True/False splits, leaves = actions), and an Evaluator that scores each root-to-leaf path.

2. A quantitative scoring rule for uncertain scenarios. Each path is scored by scenario likelihood (estimated probability the leaf's premise is true), conditional gain (how much the action advances the goal if the scenario holds), and execution cost (movement distance and message length), combined into a single utility that ranks actions under uncertainty.

3. Reframing communication as an atomic action, not the search mechanism. Unlike prior tree-search multi-agent planners that treat dialogue as the substrate of search, PCE places communication in the same candidate action space as physical actions, so the agent communicates only when it scores higher than acting alone.

4. Empirical validation across benchmarks, backbones, ablations, and a human user study. Evaluation on C-WAH and TDW-MAT with three LLM backbones, component ablations, scaling ablations, and a 12-participant user study comparing PCE against a no-communication and an always-communicate variant.

Main Findings

• PCE beats communication-centric baselines on task performance. On C-WAH (Total Steps, lower is better), PCE with GPT-4o mini achieved 42.76 steps versus CoELA 60.40, REVECA 46.80, CaPo 60.82, and CoTS 64.00. With GPT-OSS:20B it achieved 49.60 steps (CoELA 72.72, REVECA 53.86, CaPo 68.34, CoTS 65.26), and with Gemma3:4B 59.20 steps (CoELA 77.20, REVECA 62.56, CaPo 75.88, CoTS 72.32).

• The same pattern holds on TDW-MAT success rates. With GPT-4o mini, PCE reached Total 87.50 versus CoELA 62.50, REVECA 81.25, CaPo 73.33, and CoTS 75.00, with Food 89.17 and Stuff 85.83. On GPT-OSS:20B PCE scored Total 81.25 (CoELA 55.00, REVECA 73.33, CaPo 65.41, CoTS 59.17), and on Gemma3:4B Total 70.83 (CoELA 45.84, REVECA 52.09, CaPo 67.50, CoTS 63.33).

• Communication drops sharply. In C-WAH with GPT-4o mini, PCE logged 1.70 communication actions versus 9.88 (CoELA), 6.00 (REVECA), 8.72 (CaPo), and 10.24 (CoTS). The paper explicitly treats Comm as a descriptive diagnostic rather than a success metric, since communication can either waste time or prevent false plans.

• Token usage is comparable, not uniformly lower. The abstract and conclusion describe PCE as showing "comparable token usage." In C-WAH with GPT-4o mini, PCE's Usages was 44353.56, above CaPo's 41702.00 but below CoELA's 55467.12; in C-WAH with Gemma3:4B, PCE's 50984.7 was above REVECA's 44637.58; in TDW-MAT with GPT-4o mini, PCE's 197807.29 was above CoELA's 113058.83 and REVECA's 185453.54. The paper attributes this to PCE's three-module architecture having higher per-step inference cost, offset by substantially shorter episodes.

• Every component matters. In the C-WAH ablation with GPT-4o mini, removing any module worsened Total Steps: full PCE 42.76, w/o Planner 56.46, w/o Composer 46.82, w/o Evaluator 47.34. The w/o Planner variant also exploded in token cost to 139918.56 Usages, and w/o Composer produced almost no communication (0.26).

• Scaling alone is not enough. Comparing PCE against a "Planner only" variant while scaling Gemma3:4B from 4B → 12B → 27B and GPT-OSS:20B's reasoning depth from Low → Medium → High, the paper reports that Planner only showed only modest improvements, while PCE consistently achieved faster goal completion — indicating PCE's gains are additive to scaling rather than a substitute for it.

• Humans prefer PCE's communication pattern. In a user study with 12 participants (mean age 26.8; 8 male, 4 female) in C-WAH, comparing PCE, a w/o Com variant (communication removed), and a Com always variant (communication forced before each physical action), PCE scored highest on all four 7-point Likert questions: Appropriateness, Usefulness, Efficiency, and Trust. Interview feedback said Com always disrupted workflows while w/o Com made intentions unclear and harder to trust.

• Framework generality. PCE is designed to operate on generic reasoning traces rather than model-specific internals, and improvements were observed on all three backbones tested: GPT-4o mini, GPT-OSS:20B, and Gemma3:4B.

Methodology in Plain English

The authors model the problem as a decentralized partially observable Markov decision process where each agent sees only part of the environment, agents' actions split into physical actions and communication actions, and communication is costly — each message consumes a step and arrives with a one-step delay. Messages are capped at 500 characters per frame in both environments.

Their key empirical observation is that when an LLM planner chooses an action, its reasoning trace already contains an explicit guess about the unknown — for example, "you might find something useful in this cabinet" — but these guesses are used locally and never aggregated, so the planner cannot compare competing hypotheses.

PCE fixes that in three stages. The Planner produces a candidate action plus its reasoning trace. The Composer reads the trace and the context, extracts those assumptions as decision-tree nodes with True/False branches, and expands the tree top-down, always picking the assumption that most reduces uncertainty and most affects the action choice. When no assumptions remain, it generates new ones grounded in entities already in the context, and it can introduce communication actions as leaves. Expansion stops at depth D = 3 or when further splits would not change the action.

The Evaluator then scores every root-to-leaf path. It estimates scenario likelihood and conditional gain with an LLM, defines expected gain as likelihood times gain, and defines execution cost as a weighted sum of estimated traversing distance for movement and estimated message length for communication. The final utility is expected gain minus a cost-sensitivity-weighted cost term, with all scores normalized to between 0 and 1. The leaf with the highest utility determines the agent's action.

Default hyperparameters used in all experiments are D = 3, α = 1, β = 1, λ = 1, K_action = 10, K_message = 3. Benchmarks are C-WAH (10 episodes, 3–5 sub-goals per episode, horizon 250 steps) and TDW-MAT (24 episodes, 10 target objects, 2–5 containers, horizon 3000 steps). Baselines are CoELA, REVECA, CaPo, and CoTS, all run under identical environmental and communication settings.

Why This Matters

Impact on research. The paper argues that uncertainty in embodied planning is not solved by making models bigger or reasoning chains deeper — it needs an explicit mechanism to identify and weigh assumptions. It also offers a different design point from communication-centric multi-agent planners: assumptions become first-class decision variables, and dialogue becomes just one action competing on expected utility. The paper states that, to the authors' knowledge, no prior work had systematically examined whether uncertainty in embodied planning can be resolved simply by scaling LLMs.

Potential real-world applications (the paper's experiments are confined to simulated multi-room household tasks; these are directions the framework could plausibly serve):

• Household service robots coordinating on chores like preparing a meal or setting a table, where each robot only perceives part of the home.
• Human–robot collaboration in shared workspaces, where excessive robot questioning is disruptive and silence is confusing — the exact trade-off the user study probes.
• Multi-robot teams operating under partial observability where continuous radio traffic is expensive or impractical.
• Any decentralized multi-agent system where agents must decide between physically investigating something and asking a partner about it.

Industry relevance. The Usages metric explicitly counts total token consumption across all LLM modules, treating it as a proxy for computational cost — a direct cost signal for anyone deploying LLM-driven agents. The framework's independence from model-specific internals means it can sit on top of commercial and open-source backbones alike, which the paper demonstrates with one commercial model and two open-source ones. The reliance on generic reasoning traces rather than internal activations also makes it compatible with API-only models.

Future Directions

• Extending PCE to more complex and dynamic environments than the simulated multi-room household tasks used here.
• Scaling to larger and more diverse agent teams; the paper reports a scalability test with an increasing number of agents in its appendix.
• Adaptive discovery of new assumptions in real time, rather than assumptions generated during tree expansion.
• Open questions the paper leaves implicit: why token usage is not uniformly lower than baselines despite shorter episodes, how sensitive results are to the cost-balancing constants α, β, and λ (reported in appendix sensitivity analyses), and whether the human-study findings generalize beyond the 12 participants tested.

Target Audience

Researchers and practitioners working on embodied AI, LLM-based planning, and multi-agent cooperation under partial observability. It is most useful to readers who already understand the basics of LLM prompting and are looking for a principled alternative to communication-heavy coordination, as well as to engineers deploying multi-agent LLM systems where token cost, latency, and human trust all matter. Readers wanting the full formalism — the DEC-POMDP definition, the cost and utility equations, environment details, baseline descriptions, and the additional analyses referenced as appendices A.5 through A.12 — will need the paper itself, since those appendices are not included in the provided content.

Authors’ abstract

Embodied agents operating in multi-agent, partially observable, and decentralized environments must plan and act despite pervasive uncertainty about hidden objects and collaborators' intentions. Recent advances in applying Large Language Models (LLMs) to embodied agents have addressed many long-standing challenges, such as high-level goal decomposition and online adaptation. Yet, uncertainty is still primarily mitigated through frequent inter-agent communication. This incurs substantial token and time costs, and can disrupt established workflows, when human partners are involved. We introduce PCE, a Planner-Composer-Evaluator framework that converts the fragmented assumptions latent in LLM reasoning traces into a structured decision tree. Internal nodes encode environment assumptions and leaves map to actions; each path is then scored by scenario likelihood, goal-directed gain, and execution cost to guide rational action selection without heavy communication. Across two challenging multi-agent benchmarks (C-WAH and TDW-MAT) and three diverse LLM backbones, PCE consistently outperforms communication-centric baselines in success rate and task efficiency while showing comparable token usage. Ablation results indicate that the performance gains obtained by scaling model capacity or reasoning depth persist even when PCE is applied, while PCE consistently raises the baseline across both capacity and reasoning-depth scales, confirming that structured uncertainty handling complements both forms of scaling. A user study further demonstrates that PCE produces communication patterns that human partners perceive as more efficient and trustworthy. Together, these results establish a principled route for turning latent LLM assumptions into reliable strategies for uncertainty-aware planning.