Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generation
Not on the map. Why?
Abstract (arXiv)
Modeling human-human interactions from text remains challenging because it requires not only realistic individual dynamics but also precise, text-consistent spatiotemporal coupling between agents. Currently, progress is hindered by 1) limited two-person training data, inadequate to capture the diverse intricacies of two-person interactions; and 2) insufficiently fine-grained text-to-interaction modeling, where language conditioning collapses rich, structured prompts into a single sentence embedding. To address these limitations, we propose our Text2Interact framework, designed to generate realistic, text-aligned human-human interactions through a scalable high-fidelity interaction data synthesizer and an effective spatiotemporal coordination pipeline. First, we present InterCompose, a scalable synthesis-by-composition pipeline that aligns LLM-generated interaction descriptions with strong single-person motion priors. Given a prompt and a motion for an agent, InterCompose retrieves candidate single-person motions, trains a conditional reaction generator for another agent, and uses a neural motion evaluator to filter weak or misaligned samples-expanding interaction coverage without extra capture. Second, we propose InterActor, a text-to-interaction model with word-level conditioning that preserves token-level cues (initiation, response, contact ordering) and an adaptive interaction loss that emphasizes contextually relevant inter-person joint pairs, improving coupling and physical plausibility for fine-grained interaction modeling. Extensive experiments show consistent gains in motion diversity, fidelity, and generalization, including out-of-distribution scenarios and user studies. We will release code and models to facilitate reproducibility.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 7 Oct 2025.
Builds on
41 of 49 references.
References · 41
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2022Human Motion Diffusion Modelcited by 1,062
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
- 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
29 more
- 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
- 2023Human Motion Diffusion as a Generative Priorcited by 354
- 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
- 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
- 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
- 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 221
- 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
- 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
- 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 109
- 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
- 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
- 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
- 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 66
- 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 53
- 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
- 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
- 2023Digital Life Project: Autonomous 3D Characters with Social Intelligencecited by 43
- 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
- 2024LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environmentcited by 33
- 2023PSVT: End-to-End Multi-person 3D Pose and Shape Estimation with Progressive Video Transformerscited by 31
- 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 29
- 2024in2IN: Leveraging individual Information to Generate Human INteractionscited by 29
- 2024PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videoscited by 27
- 2024Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaptioncited by 18
- 2025MOSPA: Human Motion Generation Driven by Spatial Audiocited by 11
- 2023DROP: Dynamics Responses from Human Motion Prior and Projective Dynamicscited by 10
- 2025ModSkill: Physical Character Skill Modularizationcited by 7
- 2023Coordinate Transformer: Achieving Single-stage Multi-person Mesh Recovery from Videoscited by 5
Built on by
Citing papers per year: 2025: 2; 2026: 8.
10 papers.
- 2025EgoReAct: Egocentric Video-Driven 3D Human Reaction Generationcited by 7
- 2026UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priorscited by 5
- 2025InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphscited by 4
- 2026Next-Scale Autoregressive Models for Text-to-Motion Generationcited by 4
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?cited by 1
- 2026Unified Number-Free Text-to-Motion Generation Via Flow Matchingcited by 0
- 2026Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulationcited by 0
- 2026InterCMDM: Block-Causal Diffusion for Autoregressive Human Interaction Generationcited by 0
- 2026ARMS: Anchor-Relational Motion Streaming for Seamless Solo-Social Motion Transitionscited by 0
- 2026MoSAT: Human Motion Generation from Spatial Audio and Textual Descriptioncited by 0
- 2026MoSAT: Human Motion Generation from Spatial Audio and Textual Description20 Sep 2026
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?29 Jul 2026
- 2026ARMS: Anchor-Relational Motion Streaming for Seamless Solo-Social Motion Transitions7 Jul 2026
- 2026InterCMDM: Block-Causal Diffusion for Autoregressive Human Interaction Generation2 Jul 2026
- 2026Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation22 Apr 2026
- 2026Next-Scale Autoregressive Models for Text-to-Motion Generation4 Apr 2026
- 2026Unified Number-Free Text-to-Motion Generation Via Flow Matching27 Mar 2026
- 2026UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors16 Mar 2026
- 2025EgoReAct: Egocentric Video-Driven 3D Human Reaction Generation28 Dec 2025
- 2025InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphs8 Dec 2025