Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animation

AI summary

Overview

Research area: computer vision and 3D human motion generation, specifically human-human interaction animation from static poses and images.

Technical level: Advanced — the paper assumes familiarity with diffusion models, SMPLX body representations, and motion-generation benchmarks.

One-sentence scope: The paper introduces Ponimator, a framework that treats a pair of close-proximity "interactive poses" as an anchor and learns to unfold them into short interaction motion clips, enabling two-person image animation, single-person interaction generation, and text-to-interaction synthesis.

What This Paper Is About

Most systems that animate people in images either reconstruct a static two-person pose or generate video that loses interaction coherence, and most two-person motion generators follow text without reliably producing close physical contact. The authors observe that a pair of poses in close contact inherently encodes spatial relationships, constraints, and intent, and therefore proposes to use that interactive pose as an intermediate anchor from which past and future motion can be generated. The goal is a single framework that both generates interactive poses and animates them into motion across image, pose, and text inputs.

Key Contributions

1. Ponimator, a framework that learns the dynamics prior of interactive poses from motion capture data, focused on proximal human-human interaction animation.
2. Evidence that the learned prior generalizes to poses extracted from open-world images, enabling animation of social interactions in real photographs.
3. An interactive pose generator that produces two-person poses from a single-person pose, text, or both, which combined with the animator supports reaction animation and text-to-interaction synthesis.
4. Demonstration that the two components are bridged by interactive poses, giving a simple and interpretable formulation that avoids physical simulators or intricate model designs.

Main Findings

• Interactive pose as anchor works. In unconstrained interaction synthesis on the Inter-X dataset, the method reaches FID 22.6 and a contact ratio of 68.1, against ComMDM (88.8 / 44.3), RIG (65.2 / 44.3), InterGen (56.6 / 50.9), and an adapted MDM* (62.6 / 66.4). Ground truth is FID 0.3 and contact ratio 70.6.
• Ablations confirm each component matters. On Inter-X interactive pose animation, removing the anchor raises FID from 5.0 to 7.1, removing time encoding gives 6.3, removing joints conditioning gives 5.6, and using random poses as anchors gives 5.8; the full model reaches 5.0. On Dual-Human the same pattern holds (ours 24.2 versus w/o anchor 36.9, - time 30.3, - joints 29.9, random-pose 30.1).
• Text-to-interaction synthesis improves. On Inter-X, the method achieves FID 1.82, MModality 1.46, and contact ratio 45.9, versus InterGen (2.87, 1.42, 39.8) and an end-to-end no-anchor baseline (2.74, 1.41, 39.0). Ground truth FID is 0.06.
• Single pose-to-interaction synthesis improves. On Inter-X, FID drops to 27.8 with Precision 0.91, Recall 0.48, and contact ratio 73.3, compared with the no-anchor baseline at FID 40.0, Precision 0.87, Recall 0.43, and contact ratio 67.5. Ground truth is FID 0.3, Precision 1.0, Recall 1.0, contact ratio 70.6.
• The prior is universal. Qualitative results in Figure 7 span in-domain datasets (Inter-X, Dual-Human) and out-of-domain datasets (Duolando, Hi4D, Interhuman), and the model generates interactions involving more than two people without modification or retraining.
• Open-world image animation works. Interactive poses extracted from the FlickrCI3D dataset using an off-the-shelf estimator are animated into realistic motion (Figure 5), and single-person results are shown on Motion-X (Figure 6).
• Generated motion supports video synthesis. Generated interactive motions are used as intermediate results; an off-the-shelf human reconstruction model recovers textured humans from a single image, which are paired with an arbitrary second person's texture to produce interaction videos.
• Reported limitations. The method focuses on short interaction segments, relies solely on human poses and ignores scene context, can produce contact errors and foot sliding from pose inaccuracies, and close interactions may lead to inter-person penetration.

Methodology in Plain English

The authors define an "interactive pose" as two people whose SMPLX body models are in proximity and close contact. From the Inter-X and Dual-Human motion capture datasets, they detect interactive poses by spatial proximity (using a contact threshold on SMPL-X vertices), then cut out the pose plus its past and future frames to form a 3-second interaction clip. Their key modeling choice is to factor the interaction motion as a product of two terms: one describing the dynamics given the interactive pose, and one describing the interactive pose itself.

The dynamics term is handled by a diffusion model (built on the DiT architecture with stacked Transformer blocks alternating spatial attention for contact and temporal attention for motion). Instead of predicting full poses, it predicts motion residuals relative to the interactive pose, and the interactive frame is held fixed during sampling via an imputation step, with the interaction time index injected as a one-hot condition. Conditions are encoded by feeding SMPLX forward kinematics joint positions through an MLP and injecting them via AdaIN. Training combines a diffusion loss, an SMPL loss, an interaction loss, and a velocity loss, with all weights set to 1 except the interaction loss at 0.5, plus Gaussian noise augmentation (scale 0.02) on the interactive pose to handle real-world noise.

The pose-generation term is a second conditional diffusion model that produces an interactive pose from a single-person pose, text, or both. Text is encoded with a frozen CLIP-ViTL/14 encoder and injected via AdaLN; the model keeps spatial attention and removes temporal attention. Because SMPL shapes are tied to gender labels, the model instead predicts global joint positions of the rest pose, from which body shape is recovered by inverse kinematics. Masking of the text and pose conditions (sampled with p_text = 0.8 and p_pose = 0.2 during training) lets one model handle all input combinations.

For applications, a two-person image is converted to an interactive pose with an off-the-shelf estimator and animated; a single-person image has its pose estimated, a partner pose generated, and then both are animated; and a text prompt goes straight to the pose generator before animation. The interaction time index I controls the animation type: I = 0 predicts future motion, I = N reconstructs the past, and N/2 gives symmetric animation.

Why This Matters

The paper reframes human-human interaction generation as unfolding a static prior rather than learning dynamics from scratch or fine-tuning single-person priors, and it shows that this simple anchor produces better contact and motion quality than larger end-to-end pipelines. It also demonstrates that priors learned from high-quality motion capture transfer to noisy in-the-wild image poses, which is a meaningful bridge between mocap research and real photography.

Real-world applications:

• Animating social photographs, such as turning a still image of two people into a short hug, handshake, or combat clip.
• Reacting to a single-person photo by generating a plausible partner and animating the pair.
• Generating short two-person interaction clips directly from text prompts for previsualization or content creation.
• Producing stable motion intermediates for pose-guided video diffusion or texture-based human video synthesis.

Industry relevance: the work comes from Snap Inc. (with a co-author at the University of Illinois Urbana-Champaign), and the application set — animating user photos, generating interaction videos, and text-driven interaction synthesis — maps directly onto social camera and augmented reality content pipelines.

Future Directions

• Extending from short interaction segments to longer interaction sequences, which the authors explicitly list as a limitation.
• Incorporating scene context, since the current method relies solely on human poses and ignores the environment.
• Reducing contact errors, foot sliding, and inter-person penetration caused by inaccurate input poses.
• Exploring how the universal prior can be extended further, given that it already generalizes to unseen subjects, out-of-domain datasets, and interactions beyond two people without retraining.

Target Audience

Researchers and practitioners in computer vision and graphics working on human motion generation, human-human interaction modeling, image animation, and diffusion-based generative models. It is also relevant to engineers building social media photo and video animation features, and to readers interested in how simple, interpretable priors can outperform more complex end-to-end designs.

Authors’ abstract

Close-proximity human-human interactive poses convey rich contextual information about interaction dynamics. Given such poses, humans can intuitively infer the context and anticipate possible past and future dynamics, drawing on strong priors of human behavior. Inspired by this observation, we propose Ponimator, a simple framework anchored on proximal interactive poses for versatile interaction animation. Our training data consists of close-contact two-person poses and their surrounding temporal context from motion-capture interaction datasets. Leveraging interactive pose priors, Ponimator employs two conditional diffusion models: (1) a pose animator that uses the temporal prior to generate dynamic motion sequences from interactive poses, and (2) a pose generator that applies the spatial prior to synthesize interactive poses from a single pose, text, or both when interactive poses are unavailable. Collectively, Ponimator supports diverse tasks, including image-based interaction animation, reaction animation, and text-to-interaction synthesis, facilitating the transfer of interaction knowledge from high-quality mocap data to open-world scenarios. Empirical experiments across diverse datasets and applications demonstrate the universality of the pose prior and the effectiveness and robustness of our framework.