OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization
Not on the map. Why?
Abstract (arXiv)
Socially intelligent AI systems must reason across diverse human behavioral tasks and generalize to new social contexts. However, behavioral data is inherently heterogeneous, comprising diverse modalities and prediction targets that produce uneven training signals across samples, creating imbalanced learning dynamics that challenge existing AI models. To address this, we develop Omnisapiens-7B 2.0, a foundation model for social behavior processing that explicitly addresses learning from heterogeneous behavioral data. This is enabled through Heterogeneity-Aware Relative Policy Optimization, a new RL method that rebalances learning signals across samples by approximating each sample's contribution to the policy update and using these estimates to drive geometrically centered, inertially smoothed advantage modulation for stable training. Omnisapiens-7B 2.0 achieves the best and most consistent performance across 10 behavioral tasks, while also attaining the best performance on all five held-out benchmarks, with gains of up to +12.02% and +9.37% respectively. Furthermore, it demonstrates more consistent and interpretable reasoning traces, supporting reliable real-world behavioral applications. Our model is available at https://github.com/MIT-MI/human_behavior_atlas.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 45 of its 78 references that are in the map. First posted on arXiv 11 Feb 2026.
Builds on
45 of 78 references.
References · 45
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2015Trust Region Policy Optimizationcited by 2,503
- 2025Gemma 3 Technical Reportcited by 1,686
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2017Deep Reinforcement Learning that Matterscited by 727
- 2025Qwen2.5-Omni Technical Reportcited by 704
33 more
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2014Convex Optimization: Algorithms and Complexitycited by 569
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2025Qwen3-Omni Technical Reportcited by 451
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
- 2017Distral: Robust Multitask Reinforcement Learningcited by 231
- 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 193
- 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
- 2019Towards Multimodal Sarcasm Detection (An _Obviously_ Perfect Paper)cited by 173
- 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 147
- 2019UR-FUNNY: A Multimodal Language Dataset for Understanding Humorcited by 130
- 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
- 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
- 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
- 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
- 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Modelscited by 81
- 2018Composable Deep Reinforcement Learning for Robotic Manipulationcited by 70
- 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 68
- 2022Evaluation of post-hoc interpretability methods in time-series classificationcited by 53
- 20244M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalitiescited by 52
- 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 52
- 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 50
- 2022Automatic Depression Detection: An Emotional Audio-Textual Corpus and a GRU/BiLSTM-based Modelcited by 37
- 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?cited by 27
- 2024Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusioncited by 18
- 2024Explainable Natural Language Processing for Corporate Sustainability Analysiscited by 9
- 2025MimeQA: Towards Socially-Intelligent Nonverbal Foundation Modelscited by 9
- 2023MMASD: A Multimodal Dataset for Autism Intervention Analysiscited by 8
- 2022PTSD in the Wild: A Video Database for Studying Post-Traumatic Stress Disorder Recognition in Unconstrained Environmentscited by 6
- 2024Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognitioncited by 5
- 2024Revisiting the robustness of post-hoc interpretability methodscited by 5
- 2025Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understandingcited by 5
- 2021A Novel Markovian Framework for Integrating Absolute and Relative Ordinal Emotion Informationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Social-R1: Towards Human-like Social Reasoning in LLMs
- 2025To Mask or to Mirror: Human-AI Alignment in Collective Reasoning
- 2026Beyond Right and Wrong: Evaluating Second-order Social Reasoning in Large Language Models
- 2025Behavioral Inference at Scale: The Fundamental Asymmetry Between Motivations and Belief Systems
- 2025Social World Models
- 2026Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents
- 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback
- 2025DaGRPO: Rectifying Gradient Conflict in Reasoning via Distinctiveness-Aware Group Relative Policy Optimization