# Inter-X: Towards Versatile Human-Human Interaction Analysis

arXiv: https://arxiv.org/abs/2312.16051 · HTML: https://mlsanity.com/paper/2312.16051/

## Key facts

- Cited by 98 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 104 references that are in the map.
- First posted on arXiv 26 Dec 2023.

## Builds on (most cited first, 30 of 69)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019)

## Built on by (most cited first, 30 of 98)

- [MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model](https://mlsanity.com/paper/2404.19759/) (2024)
- [Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression](https://mlsanity.com/paper/2411.16575/) (2024)
- [Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data](https://mlsanity.com/paper/2507.07095/) (2025)
- [ReGenNet: Towards Human Action-Reaction Synthesis](https://mlsanity.com/paper/2403.11882/) (2024)
- [ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model](https://mlsanity.com/paper/2412.14559/) (2024)
- [InterMask: 3D Human Interaction Generation via Collaborative Masked Modeling](https://mlsanity.com/paper/2410.10010/) (2024)
- [HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects](https://mlsanity.com/paper/2407.12371/) (2024)
- [InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2509.09555/) (2025)
- [Absolute Coordinates Make Motion Generation Easy](https://mlsanity.com/paper/2505.19377/) (2025)
- [Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation](https://mlsanity.com/paper/2502.20370/) (2025)
- [TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generation](https://mlsanity.com/paper/2408.17135/) (2024)
- [A Survey on Human Interaction Motion Generation](https://mlsanity.com/paper/2503.12763/) (2025)
- [The Quest for Generalizable Motion Generation: Data, Model, and Evaluation](https://mlsanity.com/paper/2510.26794/) (2025)
- [Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis](https://mlsanity.com/paper/2508.02106/) (2025)
- [SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters](https://mlsanity.com/paper/2412.00174/) (2024)
- [MMHead: Towards Fine-grained Multi-modal 3D Facial Animation](https://mlsanity.com/paper/2410.07757/) (2024)
- [SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidance](https://mlsanity.com/paper/2503.01291/) (2025)
- [HERO: Human Reaction Generation from Videos](https://mlsanity.com/paper/2503.08270/) (2025)
- [Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes](https://mlsanity.com/paper/2410.10790/) (2024)
- [MotionBank: A Large-scale Video Motion Benchmark with Disentangled Rule-based Annotations](https://mlsanity.com/paper/2410.13790/) (2024)
- [SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis](https://mlsanity.com/paper/2412.20104/) (2024)
- [SocialGen: Modeling Multi-Human Social Interaction with Language Models](https://mlsanity.com/paper/2503.22906/) (2025)
- [LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokens](https://mlsanity.com/paper/2602.12370/) (2026)
- [Co3Gesture: Towards Coherent Concurrent Co-speech 3D Gesture Generation with Interactive Diffusion](https://mlsanity.com/paper/2505.01746/) (2025)
- [MVHumanNet++: A Large-scale Dataset of Multi-view Daily Dressing Human Captures with Richer Annotations for 3D Human Digitization](https://mlsanity.com/paper/2505.01838/) (2025)
- [PINO: Person-Interaction Noise Optimization for Long-Duration and Customizable Motion Generation of Arbitrary-Sized Groups](https://mlsanity.com/paper/2507.19292/) (2025)
- [You Think, You ACT: The New Task of Arbitrary Text to Motion Generation](https://mlsanity.com/paper/2404.14745/) (2024)
- [Multi-Person Interaction Generation from Two-Person Motion Priors](https://mlsanity.com/paper/2505.17860/) (2025)
- [Motion Generation: A Survey of Generative Approaches and Benchmarks](https://mlsanity.com/paper/2507.05419/) (2025)
- [Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions](https://mlsanity.com/paper/2508.04681/) (2025)

## Abstract (arXiv)

The analysis of the ubiquitous human-human interactions is pivotal for understanding humans as social beings. Existing human-human interaction datasets typically suffer from inaccurate body motions, lack of hand gestures and fine-grained textual descriptions. To better perceive and generate human-human interactions, we propose Inter-X, a currently largest human-human interaction dataset with accurate body movements and diverse interaction patterns, together with detailed hand gestures. The dataset includes ~11K interaction sequences and more than 8.1M frames. We also equip Inter-X with versatile annotations of more than 34K fine-grained human part-level textual descriptions, semantic interaction categories, interaction order, and the relationship and personality of the subjects. Based on the elaborate annotations, we propose a unified benchmark composed of 4 categories of downstream tasks from both the perceptual and generative directions. Extensive experiments and comprehensive analysis show that Inter-X serves as a testbed for promoting the development of versatile human-human interaction analysis. Our dataset and benchmark will be publicly available for research purposes.