TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size

AI summary

Overview

Research area: Physics-based humanoid control and cooperative human-object interaction (HOI), spanning reinforcement learning, computer graphics, and robotics.

Technical level: Advanced. The paper assumes familiarity with reinforcement learning (PPO), adversarial motion priors (AMP), Transformer architectures, and physics simulation.

Scope: This paper presents a unified decentralized control framework that lets a single trained policy coordinate teams of 2–8 physics-simulated humanoids as they jointly lift and carry tables of varying shapes and weights.

What This Paper Is About

Existing physics-based humanoid controllers work well for a single agent, but extending them to cooperative tasks—such as several humanoids jointly lifting a heavy table—runs into two walls: most policies are built on fixed-size MLP inputs, so they cannot handle teams of different sizes, and they rely on multi-person reference motion data that barely exists. TeamHOI solves both problems by training one Transformer-based policy that scales to any team size and by adapting single-human motion data through a masked motion prior so that cooperation can emerge without multi-human demonstrations.

Key Contributions

1. A framework that trains a single decentralized policy capable of driving any number of cooperating humanoid agents, with each agent acting only on local observations.

2. A Transformer-based policy network that includes teammate tokens, letting each agent attend to its teammates' states through cross-attention layers that scale gracefully as team size grows.

3. A masked Adversarial Motion Prior strategy that uses single-human reference motions while masking object-interacting body parts (hands, forearms), allowing task rewards to guide hand-object interaction and unlocking more diverse cooperative skills.

4. A formation reward that combines angular-spread and principal-axes coverage terms, agnostic to both the number of agents and the shape of the carried object, encouraging stable self-organized carrying formations.

Main Findings

• Success across team sizes: A single TeamHOI policy achieves 99.1%, 99.2%, and 97.5% success rates for 2-, 4-, and 8-agent carrying tasks, whereas baselines adapted from CooHOI degrade sharply outside their training team size (e.g., CooHOI*-2 drops to 10.1% at 8 agents).

• Heavy-load generalization: Under a 5× table-weight condition, only TeamHOI with 8 agents achieves meaningful cooperation (81.1% success); baselines reach at most 14.2% and often fail entirely.

• Better cooperation quality: TeamHOI maintains contact between agents and the object for 90–96% of the transport window, compared with 1–82% for baselines, indicating genuinely collective rather than fragmented behavior.

• Smoother motion: Mean absolute jerk of contact-point trajectories is 34–51 m/s³ for TeamHOI versus 36–190 m/s³ for baselines, showing more stable and synchronized transport.

• Masked AMP is necessary: Ablation shows that removing the masked discriminator causes conflicting objectives between motion realism and object interaction, drastically lowering lifting-stage success.

• Principal-axes reward matters: Without the coverage reward, agents drift into unnatural diagonal stepping patterns and the table rotates excessively; with it, teams align along the object's natural axes of rotational stability.

• Team-size-aware advantage normalization: Normalizing PPO advantages separately per team size is required for stable mixed-size training; global normalization yields lower task reward.

Methodology in Plain English

The team trains humanoid agents in the IsaacGym physics simulator using reinforcement learning. Each agent sees only its own body state plus limited local cues: where the table is, where 64 candidate grip points lie along its edge, where the goal is, and a small set of teammate cues (position, heading, angular offset around the table). These cues become "teammate tokens" fed into a Transformer backbone that alternates self-attention over the agent's own tokens with cross-attention over teammates, so the same network handles 2, 4, or 8 teammates without changing architecture.

To keep motion natural without multi-human reference data, the team runs two AMP discriminators. The full-body one judges the whole pose against single-human walking and pickup clips from the AMASS dataset; the masked one ignores hands and forearms. A sigmoid gate blends the two style rewards based on how close the agent is to the object, so when the agent touches the table, only the masked prior applies and the hands are free to be shaped by task rewards instead.

The carrying task itself is staged: walk to the table, assume a stable formation, lift, transport to a target, and set down. To make formations emerge without hand-assigned contact points, the authors add two rewards. The angular-spread reward pushes agents toward equal angular spacing (2π/m) around the table. The principal-axes coverage reward projects agents onto the table perimeter, builds a support polygon, and measures how well that polygon spans the table's principal axes through its center—so agents naturally align along stable, symmetric directions. IsaacGym's fixed-actor constraint is worked around by placing unused humanoids on a dummy ceiling plane where they are excluded from rewards, observations, and gradients.

Why This Matters

Impact on research: The paper removes the fixed-team-size assumption that has constrained physics-based multi-humanoid control, and it shows that cooperative HOI skills can be learned from single-human motion data alone. Any team-size unified training and masked AMP are both reusable techniques that could transfer to other multi-agent embodied tasks.

Real-world applications:

• Multi-character animation for films, games, and virtual production, where teams of virtual humans must physically coordinate around shared props.
• Robotics for warehouse and construction settings, where groups of legged or humanoid robots must jointly manipulate large objects.
• Simulation-based training for sports teams and industrial crews, where coordinated physical effort under varying group sizes matters.
• Interactive VR/AR worlds with physically grounded NPC teams that adapt to however many characters are present.

Industry relevance: Studios and game engines need scalable character controllers that do not require retraining per scene; robotics companies moving toward multi-robot manipulation would benefit from the decentralized, communication-light policy design; and simulation platforms gain a template for mixed-size multi-agent training.

Future Directions

• Extending the teammate-token scheme beyond position and heading to richer communication, such as intent or force signals, to handle tasks where physical cues alone are insufficient.

• Applying masked AMP to other interaction types (tool use, deformable objects, uneven terrain) where single-human priors still mismatch the task.

• Closing the sim-to-real gap so humanoid robot teams can demonstrate the same any-team-size cooperation on hardware.

• Exploring adaptive team formation for irregular objects and non-uniform mass distributions—the authors mention a generalized coverage formulation in supplementary material but do not evaluate it extensively.

Target Audience

Researchers and graduate students in physics-based character animation, reinforcement learning for embodied agents, and multi-agent robotics. Practitioners building multi-character simulation systems or multi-robot coordination stacks will also find the architecture and reward design directly informative. Readers should be comfortable with PPO, adversarial motion priors, and Transformer attention mechanisms.

Authors’ abstract

Physics-based humanoid control has achieved remarkable progress in enabling realistic and high-performing single-agent behaviors, yet extending these capabilities to cooperative human-object interaction (HOI) remains challenging. We present TeamHOI, a framework that enables a single decentralized policy to handle cooperative HOIs across any number of cooperating agents. Each agent operates using local observations while attending to other teammates through a Transformer-based policy network with teammate tokens, allowing scalable coordination across variable team sizes. To enforce motion realism while addressing the scarcity of cooperative HOI data, we further introduce a masked Adversarial Motion Prior (AMP) strategy that uses single-human reference motions while masking object-interacting body parts during training. The masked regions are then guided through task rewards to produce diverse and physically plausible cooperative behaviors. We evaluate TeamHOI on a challenging cooperative carrying task involving two to eight humanoid agents and varied object geometries. Finally, to promote stable carrying, we design a team-size- and shape-agnostic formation reward. TeamHOI achieves high success rates and demonstrates coherent cooperation across diverse configurations with a single policy.