# Social Structure Matters in 3D Human-Human Interaction Generation

arXiv: https://arxiv.org/abs/2606.24255 · HTML: https://mlsanity.com/paper/2606.24255/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 55 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 30 of 51)

- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [Guided Motion Diffusion for Controllable Human Motion Synthesis](https://mlsanity.com/paper/2305.12577/) (2023)
- [Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory](https://mlsanity.com/paper/2203.13055/) (2022)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [Qwen3.5-Omni Technical Report](https://mlsanity.com/paper/2604.15804/) (2026)
- [AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism](https://mlsanity.com/paper/2309.00796/) (2023)
- [Inter-X: Towards Versatile Human-Human Interaction Analysis](https://mlsanity.com/paper/2312.16051/) (2023)
- [Human Motion Generation: A Survey](https://mlsanity.com/paper/2307.10894/) (2023)
- [ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generation](https://mlsanity.com/paper/2203.07706/) (2022)
- [TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration](https://mlsanity.com/paper/2304.02419/) (2023)
- [ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions](https://mlsanity.com/paper/2311.17057/) (2023)
- [MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understanding](https://mlsanity.com/paper/2410.21747/) (2024)
- [Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation](https://mlsanity.com/paper/2305.09662/) (2023)
- [ReGenNet: Towards Human Action-Reaction Synthesis](https://mlsanity.com/paper/2403.11882/) (2024)
- [Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment](https://mlsanity.com/paper/2403.18811/) (2024)
- [InterMask: 3D Human Interaction Generation via Collaborative Masked Modeling](https://mlsanity.com/paper/2410.10010/) (2024)
- [Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions](https://mlsanity.com/paper/1912.10150/) (2019)
- [The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion](https://mlsanity.com/paper/2412.10523/) (2024)
- [Interaction Transformer for Human Reaction Generation](https://mlsanity.com/paper/2207.01685/) (2022)
- [MotionGPT3: Human Motion as a Second Modality](https://mlsanity.com/paper/2506.24086/) (2025)
- [HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation](https://mlsanity.com/paper/2512.23464/) (2025)
- [in2IN: Leveraging individual Information to Generate Human INteractions](https://mlsanity.com/paper/2404.09988/) (2024)

## Built on by (most cited first, 1 of 1)

- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)

## Abstract (arXiv)

Although text-to-motion generation has achieved strong progress in synthesizing realistic single-person motions from language, extending it to text-driven 3D human-human interaction (HHI) remains non-trivial, as HHI requires modeling the underlying social structure that governs phase progression, actor roles, and inter-actor coordination. In this paper, we formulate HHI generation as a social structure modeling and grounding problem: the model must first infer how an interaction unfolds and how the two actors coordinate their roles, and then realize this structure as continuous, physically plausible, and partner-aware 3D motion. To study how such structure should be modeled, we first examine the capability boundary of large language models (LLMs) for HHI generation. Our analysis shows that LLMs can textitthink by recovering phase decompositions and partner-aware roles, but cannot directly textitmove, as they fail to generate dynamic, physically plausible, and interaction-aware motion. This motivates our planner-executor paradigm, Think with LLM, Move with Motion Skill. The LLM planner converts implicit interaction semantics into motion-aligned social supervision by decomposing interactions into phases, assigning partner-aware actor roles, and aligning them with motion sequence. The motion executor then grounds the planned social structure into coordinated two-person motion by adapting a pretrained solo motion model with LoRA, previous-phase self-conditioning, and ego-relative partner conditioning. Together, our Solo-to-Social framework bridges social organization and motion realization, producing 3D HHI with improved phase consistency, role alignment, and partner-aware coordination.