# Ego-Exo4D Human Meshes Dataset: 4D Human Motion Reconstruction for Ego-Exo Captures

arXiv: https://arxiv.org/abs/2609.30187 · HTML: https://mlsanity.com/paper/2609.30187/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 30 references that are in the map.
- First posted on arXiv 24 Sep 2026.

## Builds on (most cited first, 26 of 26)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion](https://mlsanity.com/paper/2312.07531/) (2023)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [EgoMimic: Scaling Imitation Learning via Egocentric Video](https://mlsanity.com/paper/2410.24221/) (2024)
- [Decoupling Human and Camera Motion from Videos in the Wild](https://mlsanity.com/paper/2302.12827/) (2023)
- [World-Grounded Human Motion Recovery via Gravity-View Coordinates](https://mlsanity.com/paper/2409.06662/) (2024)
- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [TWIST: Teleoperated Whole-Body Imitation System](https://mlsanity.com/paper/2505.02833/) (2025)
- [SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control](https://mlsanity.com/paper/2511.07820/) (2025)
- [Ego-Body Pose Estimation via Ego-Head Pose Estimation](https://mlsanity.com/paper/2212.04636/) (2022)
- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation](https://mlsanity.com/paper/2410.11792/) (2024)
- [Estimating Body and Hand Motion in an Ego-sensed World](https://mlsanity.com/paper/2410.03665/) (2024)
- [AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control](https://mlsanity.com/paper/2505.03738/) (2025)
- [ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos](https://mlsanity.com/paper/2503.23877/) (2025)
- [World Models for Learning Dexterous Hand-Object Interactions from Human Videos](https://mlsanity.com/paper/2512.13644/) (2025)
- [Whole-Body Conditioned Egocentric Video Prediction](https://mlsanity.com/paper/2506.21552/) (2025)
- [ExpertAF: Expert Actionable Feedback from Video](https://mlsanity.com/paper/2408.00672/) (2024)
- [The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimation](https://mlsanity.com/paper/2504.08654/) (2025)
- [FIction: 4D Future Interaction Prediction from Video](https://mlsanity.com/paper/2412.00932/) (2024)
- [What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?](https://mlsanity.com/paper/2606.06627/) (2026)
- [ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos](https://mlsanity.com/paper/2604.10466/) (2026)

## Abstract (arXiv)

Ego-Exo4D is a large-scale dataset providing synchronized egocentric and multi-view exocentric video, a rich resource for skill learning and assessment, procedural activity understanding, and embodied AI. However, the dataset ships with only sparse 3D human pose annotations, and reconstructing dense human motion from its multi-view captures is nontrivial. To this end, we present Ego-Exo4D-HM, a large-scale dataset of 4D human motion reconstructions for Ego-Exo4D's captures, and release the accompanying reconstruction pipeline. The code, dataset, and documentation can be found at https://abhiram824.github.io/egoexo4d_human_meshes.