Attacca: Goal-Directed Control under State Continuity for Long-Horizon Embodied Agents

AI summary

Overview

• Research area: Embodied AI, in particular visual goal-conditioned low-level control policies for long-horizon, sequential task execution (the paper studies Minecraft via the MineStudio environment).
• Technical level: Advanced.
• Scope: The paper proposes and evaluates Attacca, a training recipe and policy architecture that keeps a single visual goal-conditioned controller from losing the next target across successive task hand-offs, when each task begins from the state left behind by the previous one.

What This Paper Is About

Existing visual goal-conditioned policies are usually trained and tested on isolated interactions where the target is already visible in the current observation. That leaves out what happens during continuous long-horizon execution: after one task finishes, the agent sits at a new position and orientation, the world may have changed, and the next target may be outside the field of view. The paper names this the "skill hand-off gap" and aims to close it so that one policy can search for, approach, and interact with each next target from whatever state the previous task produced.

Key Contributions

1. Context-decoupled goal sampling. Each demonstration trajectory is paired with a masked goal image of the same target class drawn from a different world, so that w(g) ≠ w(τ) and y(g) = y(τ). This removes scene and pose correspondence between the goal image and the observations, preventing the policy from using background context as a shortcut.
2. Goal-conditioned target grounding. A mask head predicts a dense target mask over the goal-conditioned observation patches, supervised by a per-patch binary cross-entropy term plus a soft Dice loss against renderer-derived target masks. Frames where the target is not yet visible get an all-zero mask, so the head learns both visibility and localization.
3. Behavioral-phase conditioning. A phase head predicts one of three behavioral phases (Search, Approach, Interact) from the temporal readout and modulates the readout via FiLM with stop-gradient, so the policy adapts its control as execution progresses.
4. A joint training objective and a purpose-built dataset. Action imitation, target grounding, and phase prediction are optimized together as L = L_BC + λ_target·L_target + λ_φ·L_φ. The HumanPlay dataset contains 1,160 human demonstrations (284,961 frames) recorded in 1,160 different worlds: Mine 822 episodes / 209,593 frames, Hunt 198 episodes / 47,991 frames, and Place 140 episodes / 27,377 frames, including 10% of episodes held out for validation.

Main Findings

• Short-horizon clean success. Attacca reaches 39.0% on Mine, 47.5% on Hunt, and 47.0% on Place, outperforming the strongest baseline by 1.7×, 2.4×, and 2.1× respectively. In the per-class breakdown it scores Mine ID 0.450 / OOD 0.250, Hunt ID 0.479 / OOD 0.467, and Place ID 0.550 / OOD 0.283.
• Baselines trail badly. With a current-view goal, ROCKET-1 and ROCKET-2 stay at or below 5% on Mine and Place, rising to 18–20% on Hunt because animals wander into view on their own. STEVE-1 (text) averages 0.005, 0.055, 0.000; STEVE-1 (video) 0.020, 0.040, 0.000; GROOT 0.010, 0.040, 0.000; JARVIS-VLA 0.080, 0.165, 0.085.
• Fine-tuning matters, but is not enough. Fine-tuning ROCKET-2 on the same demonstrations raises it to 0.235 (Mine), 0.185 (Hunt), and 0.225 (Place) average clean success — well above the released checkpoint at 0.005, 0.100, and 0.000 — yet still below Attacca, which uses the same data, initialization, and goals.
• Long-horizon chains. Attacca completes 54% of the Diamond Pickaxe chain, 30% of Wolf Feeding, and 28% of Nether Portal, i.e. 14–27 of 50 episodes per chain, while no baseline exceeds 4 episodes. Overall it completes 5–7 times as many chains as the strongest baseline, with up to a 7× improvement.
• Next-stage completion is high and stable. Attacca reaches 85–93% at most transitions, and the largest gap appears after mining the oak log, where it proceeds to the diamond ores in nearly all successful episodes versus one in seven for fine-tuned ROCKET-2.
• Components need each other. In the ablation (n = 50), neither residual mask feedback nor phase conditioning helps consistently alone (0.50/0.12/0.18 baseline; 0.54/0.14/0.08 with residual only; 0.42/0.28/0.06 with phase only), but combining both improves all three chains to 0.54/0.30/0.28.
• Residual feedback beats other pathways. With target supervision and phase conditioning fixed, residual injection scores 0.54/0.30/0.28, versus cross-attention 0.36/0.08/0.18, layerwise FiLM 0.42/0.10/0.18, and ControlNet 0.16/0.00/0.08.
• Robustness to the goal image. Pooled over two scenarios (n = 100), ROCKET-2 succeeds 0.75 ± 0.04 with close same-world goals but only 0.24 ± 0.04 with far same-world goals and 0.30 ± 0.05 with different-world goals; Attacca stays at 0.89 ± 0.03, 0.89 ± 0.03, and 0.81 ± 0.04. On four goal images with different backgrounds (50 episodes per goal), Attacca succeeds in 0.68–0.96 of episodes versus 0.08–0.28 for ROCKET-2.
• Failure diagnosis. In the diamond stage (Attacca 42 episodes, ROCKET-2 28), the diamond ore enters ROCKET-2's view in 27 of its 28 episodes but repeatedly leaves view; ROCKET-2 completes the stage in only 4 of those 27 episodes against 39 of 42 for Attacca, within the 600-step budget. The paper attributes this to ROCKET-2 reproducing the goal view rather than pursuing the masked target.
• Loss-weight sensitivity is modest. Varying each auxiliary weight over a 16× range with the other fixed changes Mine success by at most 4.5 points for λ_target and 6.0 points for λ_φ, within the ±6.7-point 95% binomial confidence interval at n = 200.

Methodology in Plain English

The authors keep the interface of a masked goal image — an RGB picture with a binary mask showing which object to pursue — but change where that goal comes from. Instead of using a frame from the same world the agent is in, they draw a frame of the same object class from a different recorded world, so the only usable cue is what the target looks like.

The policy (initialized from the released ROCKET-2 checkpoint) uses a frozen pretrained visual encoder to encode the goal once per task and the current observation each timestep. A goal-fusion module combines goal, mask, and observation patch features with query tokens, and a causal Transformer turns the fused sequence — together with the interaction type and previous action — into a temporal readout that an action head maps to mouse and keyboard actions.

Two auxiliary heads are added. A mask head predicts, for each observation patch, whether the target is present, trained against renderer-derived masks using per-patch binary cross-entropy plus soft Dice loss. A phase head predicts Search, Approach, or Interact, and its prediction modulates the readout through FiLM feature-wise scaling and shifting. Gradients are stopped through both heads before they influence the action pathway, so each head is trained purely by its own supervision. A residual encoder summarizes the predicted mask into a single vector that is added to the readout token before the Transformer, with its inputs detached from the computation graph.

Training data come from 1,160 human-played Minecraft demonstrations spanning the full search-to-interact progression, with per-frame target masks and phase labels (Search 33%, Approach 47%, Interact 20%; 35% of frames have no visible target). Labels are used only as training targets and never at inference. Evaluation covers short-horizon single tasks (200 episodes per method, seven seen and three held-out classes), long-horizon chains that preserve the state between stages (50 paired policy seeds per method), and ablations over components and mask-feedback pathways.

Why This Matters

The work targets a failure mode that shows up whenever autonomy is composed of sequential skills rather than a single isolated action: the controller can finish skill A but cannot start skill B because the target of B is no longer in view. Because Attacca learns to pursue the masked target itself rather than to recreate the goal image's viewpoint, it generalizes across unseen environments, held-out target classes, and longer horizons, and it does so without target coordinates, simulator state, or external grounding and navigation modules.

Real-world applications that follow from this capability:

• Household and service robots that must navigate to a cabinet, retrieve an object, carry it elsewhere, and then locate a second object, with each step starting from the pose and world state left by the previous one — the paper's own motivating example.
• Warehouse and logistics robots chaining pick, transport, and stow operations across a changing floor layout.
• Agricultural or field robots performing sequential locate-and-act operations on crops or equipment that may be out of camera view at the start of each stage.
• Simulated training pipelines and game-playing agents where long-horizon objectives require locating multiple successive targets in open-world environments.

Industry relevance centers on the skill hand-off gap: hierarchical agent stacks that pair a planner with a low-level controller have repeatedly identified the controller as a main source of failure on harder tasks, according to the paper's related-work discussion. Attacca's contribution is at the controller level, so it can plausibly be dropped into planner-plus-policy architectures — though the paper notes the planner side is left to future work.

Future Directions

• Extending Attacca to other embodied domains beyond Minecraft, as the authors explicitly leave open.
• Pairing Attacca with learned planners, rather than the fixed task chains used in the evaluation.
• Investigating why residual mask feedback outperforms richer pathways such as cross-attention, layerwise FiLM, and ControlNet, which the paper hypothesizes relates to absorbing variation in mask presence across behavioral phases.
• Addressing the evaluation limitations discussed in the paper's Section H, which the authors flag rather than resolve.

Target Audience

Researchers and engineers working on embodied agents, visuomotor control, and long-horizon task execution, especially those building hierarchical planner-plus-policy systems or training goal-conditioned policies in open-world simulators. Practitioners interested in how goal specification (text, video, current-view mask, or cross-world mask) affects robustness will find the comparative tables and the goal-image sensitivity analysis most useful. The paper assumes familiarity with behavior cloning, Transformer policies, and auxiliary-loss training, so it is best suited to readers with an intermediate-to-advanced background in machine learning.

Authors’ abstract

A central capability of embodied agents is to accomplish complex objectives through sequences of interdependent tasks. Yet existing visual goal-conditioned policies underlying these agents are typically evaluated on isolated interactions where the target is already visible, and thus do not capture the conditions that arise during continuous long-horizon task execution. In such settings, each task begins from the state left by the previous one: the agent may end at a different position and orientation, the world may have been modified, and the next interaction target may lie outside the current field of view. As a result, agents relying on such policies may struggle to proceed to the next task when they cannot ground their target in the current observation. To address this challenge, we propose Attacca, a new approach that trains visual goal-conditioned policies on complete search-to-interact trajectories using goal images decoupled from the execution environment. Attacca uses context-decoupled goal sampling to pair each demonstration with a class-compatible masked goal image from another world, removing direct scene and pose correspondence. It learns dense current-view grounding through a target-mask prediction head, providing auxiliary supervision beyond action imitation. We further introduce behavioral-phase conditioning that teaches the policy to distinguish Search, Approach, and Interact stages and adapt its control as execution progresses. We evaluate Attacca on multiple short- and long-horizon embodied tasks in Minecraft. Our method achieves 39.0-47.5% clean success, improving over the strongest baseline by 1.7-2.4x. On long-horizon tasks, it attains 54%, 30%, and 28% completion, yielding up to a 7x improvement.