OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization

AI summary

Overview

Research area: Multimodal AI foundation models and reasoning-based reinforcement learning, applied to social behavior processing (affective, cognitive, pathological, and social signal understanding).

Technical level: Intermediate. The paper assumes familiarity with policy-gradient RL (GRPO, PPO-style clipping) and multimodal LLM training, but its core idea can be understood without the math.

Scope: This paper introduces OmniSapiens-7B 2.0, a single 7B-parameter multimodal foundation model trained across 10 behavioral tasks, and HARPO, a reinforcement learning method that rebalances learning signals across heterogeneous behavioral data.

What This Paper Is About

Behavioral data spans very different modalities (text, vision, audio) and very different prediction targets (emotion classification, social inference, anxiety detection), which produces uneven training signals across samples. Existing unified models use RL methods such as GRPO, which aggregate policy updates across these heterogeneous samples without regulating how much each task or sample contributes, so optimization can become dominated by a subset of tasks and hurt multitask performance and generalization. The goal of this work is to build a foundation model for social behavior processing whose training explicitly rebalances learning across heterogeneous behavioral data.

Key Contributions

1. OmniSapiens-7B 2.0, a foundation model for unified social behavior analysis built on Qwen 2.5 Omni-7B. The authors state it is one of the first models to jointly tackle a wide range of behavioral analysis tasks spanning affective, cognitive, pathological, and social dimensions, trained on the Human Behavior Atlas benchmark (over 100k samples) across 10 tasks.
2. HARPO (Heterogeneity-Aware Relative Policy Optimization), described as a critic-free, reasoning-based RL method designed to learn from heterogeneous behavioral data by modulating advantages at the sample and task levels, using geometrically centered, inertially smoothed modulation factors.
3. Empirical validation across three dimensions: multitask capability on 10 behavioral tasks, zero-shot generalization to five held-out behavioral benchmarks, and reasoning-trace interpretability measured through automated metrics and human evaluation.
4. Ablation and mechanism analysis isolating the effects of structured modulation, sample-level modulation, and inertial control, plus an illustrative study of how HARPO affects a weak-performing task (sarcasm) across training.

Main Findings

• Multitask performance: OmniSapiens-7B 2.0 achieves the strongest overall performance across the 10 behavioral tasks, ranking in the top 2 on 8 of 10 tasks with the strongest average task performance rank (1.90). It outperforms general-purpose models (Gemma-3-4B, Gemma 4-E4B (8B), Qwen 2.5-Omni-7B, Qwen 2.5-VL-7B, Qwen 3-VL-8B-Instruct) and prior behavioral foundation models (HumanOmniV2-7B, OmniSapiens SFT, BAM, and RL). Reported improvements over prior models reach up to +12.02%.
• RL algorithm comparison: HARPO attains the best average rank (2.10) across tasks among reasoning RL methods, compared with RLOO (2.80), RE++ (4.50), GPG (2.90), GRPO (3.90), and EMAGRPO (3.10). HARPO outperforms GRPO by up to +42.29%. The paper notes GRPO, GPG, and RE++ collapse on certain tasks (e.g., sarcasm), while HARPO does not show the same degradation on weaker tasks.
• Zero-shot generalization: OmniSapiens-7B 2.0 achieves the best performance on all five held-out benchmarks — Autism Detection (AUT, 39.91), Speech Emotion Recognition (SER, 72.11), In-the-wild Depression Recognition (IDR, 72.43), Social Media Sentiment Analysis (SMSA, 58.47), and Sarcasm Incongruity Recognition (SIR, 69.27), all reported as weighted F1. Gains over prior models reach up to +9.37%. The paper notes that OmniSapiens-7B SFT and BAM are excluded from this comparison because their task-specific classifier heads structurally preclude zero-shot inference.
• Generalization link to multitask balance: Trained on the same benchmark, OmniSapiens-7B 2.0 (HARPO) generalizes better than OmniSapiens-7B RL (GRPO) on every held-out task, alongside a better average task rank (1.90 vs. 4.20). The authors suggest more uniform learning may promote transferable behavioral features, but explicitly leave a rigorous investigation to future work.
• Reasoning consistency and robustness: OmniSapiens-7B 2.0 reaches the strongest reasoning consistency (87.7, versus 66.5 for the next best) and the highest paraphrase consistency (88.4, versus 80.7 for the next best).
• Reasoning efficiency: It produces substantially more concise reasoning traces, averaging 19.86 tokens versus 57.69 for the next best model.
• Human evaluation of reasoning: Averaged across baselines, OmniSapiens-7B 2.0 was tied with or preferred over baselines in 78.9% of comparisons for specificity, 88.3% for coherence, and 99.2% for concision. Excluding ties, effective win rates were 68.5%, 85.1%, and 99.2% respectively. The paper reports that it was preferred over, or at minimum tied with, every baseline on all three criteria.
• Mechanism on a weak task: On sarcasm (SAR), scored at 53.58% under GRPO and 70.64% under HARPO, the paper shows that after step 355 HARPO's amplification of SAR's relative task advantage coincides with continued reward increases, while GRPO's unmodulated relative task advantage stays lower alongside declining rewards.
• Ablations: Average task rank degrades from 1.90 (HARPO) to 2.00 without structured modulation, 2.60 without sample-level modulation, and 2.70 without inertial control. The ablation without structured modulation produces per-task modulation factors whose geometric mean remains above 2.5, indicating sustained advantage upscaling, whereas HARPO maintains a geometric mean of 1. Tasks with larger improvements under inertial control, SAR (+7.96%) and SEN (+5.69%), also show clearer differences in their task modulation factors over training.
• Case study: On the AUT task, the paper reports that HumanOmniV2 generates expansive reasoning chains introducing behaviors without clear evidential support, OmniSapiens-7B-RL defaults to "background (no behavior)" predictions even amid salient cues, while OmniSapiens-7B 2.0 produces concise traces focused on cues directly supported by the clip.

Methodology in Plain English

The model starts from a multimodal backbone (Qwen 2.5 Omni-7B) and is trained in a single multitask setup across all behavioral tasks rather than as separate task-specific models.

The training method, HARPO, builds on GRPO. GRPO generates a group of responses per sample and normalizes each response's reward into an advantage, which then scales that response's contribution to the policy update. Because advantages scale the gradient, tasks or samples with systematically larger advantages can dominate the shared update.

HARPO's fix works in three steps:

1. Estimate contribution. It uses the average absolute group-normalized advantage as a cheap proxy for how much each sample (within a task) and each task (across tasks) contributes to the policy update. These proxies are normalized by rollout count to stay invariant to stochastic batch sampling.
2. Modulate with a geometric reference. It computes a geometric-mean reference at both the sample level and the task level, then builds modulation factors as the reciprocal of each value's ratio to that reference. Samples or tasks contributing above the reference get downscaled advantages; those below get upscaled. Because the factors come from a geometric mean, each set of factors has a geometric mean of 1, so upscaling and downscaling exactly compensate and the overall update size is not inflated or shrunk.
3. Smooth over time. Contribution signals are updated with an exponential moving average, and modulation factors — being multiplicative ratios — are smoothed with multiplicative updates rather than additive ones. This keeps modulation responding to persistent trends instead of stochastic on-policy noise.

The final HARPO objective keeps GRPO's PPO-style clipped trust-region form and KL penalty, substituting the HARPO-modulated advantage for the standard one.

Rewards combine a task reward (binary for classification, cosine-similarity-based and normalized to [0, 1] for question answering), a binary format reward for producing a reasoning trace followed by a prediction, and an overlong length penalty, with format weight 0.2 and length scale 0.75.

Evaluation covers three dimensions: multitask performance against general-purpose and specialized behavioral models plus competing RL algorithms; zero-shot generalization on five held-out datasets without further fine-tuning; and reasoning-trace interpretability using automated consistency and length metrics plus human pairwise ratings of specificity, coherence, and concision. The paper states that the reason for evaluating on held-out datasets in the reasoning analysis is to reflect realistic deployment conditions and control for training data overlap.

Why This Matters

For research, the paper argues that reasoning RL for learning heterogeneous behavioral data is underexplored, and that multitask learning remains largely underexplored for recent reasoning RL. It offers a concrete mechanism — advantage modulation driven by contribution estimates — for the general problem of imbalanced learning dynamics when one shared policy is trained on tasks with very different signal scales. It also releases the model and makes reasoning traces public, which supports reproducibility and further interpretability work.

Real-world applications the paper points to:

• Affective computing, where systems interpret emotion, arousal–valence, sentiment, and social signals.
• Behavioral health, including detection of anxiety, depression, and PTSD, and held-out clinical-adjacent tasks such as autism detection and in-the-wild depression recognition.
• Socially intelligent AI systems that must adapt to new social contexts, such as social-pragmatic reasoning over social media content and sarcasm or humor understanding.
• Deployment settings where behavioral data is scarce and model interpretability is critical, where zero-shot transfer avoids annotation costs and concise reasoning traces allow post-hoc verification of judgments.

Industry relevance: the paper positions a single deployable model as a replacement for training, storing, and deploying separate task-specific models, and emphasizes reduced inference cost (19.86 average reasoning tokens) and verification-friendly traces — both practical concerns for products in health, education, and human-facing interaction.

Future Directions

• Rigorous study of the balance–generalization link. The authors suggest more uniform learning across behavioral tasks may promote transferable behavioral features that improve generalization, but explicitly leave a more rigorous investigation of this relationship to future work.
• Extending heterogeneity-aware RL to broader settings. Since HARPO is presented as a general response to heterogeneous training signals, a natural question is whether the same modulation mechanism transfers to other multimodal or multi-objective training regimes beyond the 10 tasks studied here.
• Scaling tasks and modalities. The model is trained on 10 tasks from the Human Behavior Atlas and evaluated on 5 held-out benchmarks; whether the approach holds as task count and modality diversity grow is not established in the reported content.
• Deployment validation in high-stakes settings. The paper motivates interpretability by critical behavioral applications such as health, but clinical validation, safety boundaries, and failure modes of the model in real deployments are not reported.

Target Audience

Researchers and practitioners working on multimodal foundation models, reinforcement learning for LLM reasoning, affective computing, and behavioral health AI. It is most useful for readers interested in multitask training dynamics and reward/advantage balancing for heterogeneous data, as well as those evaluating the interpretability of model reasoning traces. Readers seeking a self-contained RL derivation will find the core formulation here, with full derivations deferred to the paper's appendices.

Authors’ abstract

Socially intelligent AI systems must reason across diverse human behavioral tasks and generalize to new social contexts. However, behavioral data is inherently heterogeneous, comprising diverse modalities and prediction targets that produce uneven training signals across samples, creating imbalanced learning dynamics that challenge existing AI models. To address this, we develop Omnisapiens-7B 2.0, a foundation model for social behavior processing that explicitly addresses learning from heterogeneous behavioral data. This is enabled through Heterogeneity-Aware Relative Policy Optimization, a new RL method that rebalances learning signals across samples by approximating each sample's contribution to the policy update and using these estimates to drive geometrically centered, inertially smoothed advantage modulation for stable training. Omnisapiens-7B 2.0 achieves the best and most consistent performance across 10 behavioral tasks, while also attaining the best performance on all five held-out benchmarks, with gains of up to +12.02% and +9.37% respectively. Furthermore, it demonstrates more consistent and interpretable reasoning traces, supporting reliable real-world behavioral applications. Our model is available at https://github.com/MIT-MI/human_behavior_atlas.