# InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation

arXiv: https://arxiv.org/abs/2509.09555 · HTML: https://mlsanity.com/paper/2509.09555/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 102 of its 130 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 30 of 102)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022)
- [DexYCB: A Benchmark for Capturing Hand Grasping of Objects](https://mlsanity.com/paper/2104.04631/) (2021)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [OmniControl: Control Any Joint at Any Time for Human Motion Generation](https://mlsanity.com/paper/2310.08580/) (2023)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [Stochastic Scene-Aware Motion Prediction](https://mlsanity.com/paper/2108.08284/) (2021)
- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)

## Built on by (most cited first, 30 of 32)

- [InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2602.06035/) (2026)
- [FrankenMotion: Part-level Human Motion Generation and Composition](https://mlsanity.com/paper/2601.10909/) (2026)
- [Unleashing Guidance Without Classifiers for Human-Object Interaction Animation](https://mlsanity.com/paper/2603.25734/) (2026)
- [UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors](https://mlsanity.com/paper/2603.15975/) (2026)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesis](https://mlsanity.com/paper/2607.17097/) (2026)
- [Decoupled Generative Modeling for Human-Object Interaction Synthesis](https://mlsanity.com/paper/2512.19049/) (2025)
- [ViHOI: Human-Object Interaction Synthesis with Visual Priors](https://mlsanity.com/paper/2603.24383/) (2026)
- [SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework](https://mlsanity.com/paper/2605.20373/) (2026)
- [HandX: Scaling Bimanual Motion and Interaction Generation](https://mlsanity.com/paper/2603.28766/) (2026)
- [Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors](https://mlsanity.com/paper/2603.29655/) (2026)
- [Seeing Without Eyes: 4D Human-Scene Understanding from Wearable IMUs](https://mlsanity.com/paper/2604.21926/) (2026)
- [Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interaction](https://mlsanity.com/paper/2604.27491/) (2026)
- [Dex2HOI: Dexterous Bimanual Two-Object Interaction Generation](https://mlsanity.com/paper/2605.30444/) (2026)
- [HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generation](https://mlsanity.com/paper/2608.15141/) (2026)
- [SyncMV4D: Synchronized Multi-view Joint Diffusion of Appearance and Motion for Hand-Object Interaction Synthesis](https://mlsanity.com/paper/2511.19319/) (2025)
- [Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation](https://mlsanity.com/paper/2604.20336/) (2026)
- [InterPhys: Physics-aware Human Motion Synthesis in a Dynamic Scene](https://mlsanity.com/paper/2605.01036/) (2026)
- [GIRAF: Towards Generalizable Human Interactions with Articulated Objects](https://mlsanity.com/paper/2607.07880/) (2026)
- [HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction](https://mlsanity.com/paper/2608.16222/) (2026)
- [Hoi3DGen: Generating High-Quality Human-Object-Interactions in 3D](https://mlsanity.com/paper/2603.12126/) (2026)
- [LEXIS: LatEnt ProXimal Interaction Signatures for 3D HOI from an Image](https://mlsanity.com/paper/2604.20800/) (2026)
- [Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllers](https://mlsanity.com/paper/2605.11369/) (2026)
- [Policy-as-Data: Learning Generalizable HOI Diffusion Models from Simulated Physics](https://mlsanity.com/paper/2606.22806/) (2026)
- [IMAGIN-4D: Image-Guided Controllable Interaction Generation](https://mlsanity.com/paper/2606.23675/) (2026)
- [IMU-HOI: A Symbiotic Framework for Coherent Human-Object Interaction and Motion Capture via Contact-Conscious Inertial Fusion](https://mlsanity.com/paper/2606.28604/) (2026)
- [RF-HOI: Recognize Human-Object Interaction with Radio Frequency Signals](https://mlsanity.com/paper/2608.00289/) (2026)
- [PhotoHOI: Synthesizing 3D Hand-Object Interactions from a Single RGB Photograph](https://mlsanity.com/paper/2608.01905/) (2026)
- [Identity-Aware Human-Object Interaction Motion Captioning](https://mlsanity.com/paper/2608.20690/) (2026)
- [Harnessing Coupled Stream Completion For Human-Object Interaction Modeling](https://mlsanity.com/paper/2609.32551/) (2026)

## Abstract (arXiv)

While large-scale human motion capture datasets have advanced human motion generation, modeling and generating dynamic 3D human-object interactions (HOIs) remain challenging due to dataset limitations. Existing datasets often lack extensive, high-quality motion and annotation and exhibit artifacts such as contact penetration, floating, and incorrect hand motions. To address these issues, we introduce InterAct, a large-scale 3D HOI benchmark featuring dataset and methodological advancements. First, we consolidate and standardize 21.81 hours of HOI data from diverse sources, enriching it with detailed textual annotations. Second, we propose a unified optimization framework to enhance data quality by reducing artifacts and correcting hand motions. Leveraging the principle of contact invariance, we maintain human-object relationships while introducing motion variations, expanding the dataset to 30.70 hours. Third, we define six benchmarking tasks and develop a unified HOI generative modeling perspective, achieving state-of-the-art performance. Extensive experiments validate the utility of our dataset as a foundational resource for advancing 3D human-object interaction generation. To support continued research in this area, the dataset is publicly available at https://github.com/wzyabcas/InterAct, and will be actively maintained.