# Everybody Tracking Every Body

arXiv: https://arxiv.org/abs/2608.29927 · HTML: https://mlsanity.com/paper/2608.29927/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 34 references that are in the map.
- First posted on arXiv 30 Aug 2026.

## Builds on (most cited first, 30 of 31)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [OmniControl: Control Any Joint at Any Time for Human Motion Generation](https://mlsanity.com/paper/2310.08580/) (2023)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [Decoupling Human and Camera Motion from Videos in the Wild](https://mlsanity.com/paper/2302.12827/) (2023)
- [EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices](https://mlsanity.com/paper/2112.07642/) (2021)
- [World-Grounded Human Motion Recovery via Gravity-View Coordinates](https://mlsanity.com/paper/2409.06662/) (2024)
- [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024)
- [TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments](https://mlsanity.com/paper/2306.02850/) (2023)
- [You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions](https://mlsanity.com/paper/1904.09882/) (2019)
- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)
- [Flexible Motion In-betweening with Diffusion Models](https://mlsanity.com/paper/2405.11126/) (2024)
- [SAM 3D Body: Robust Full-Body Human Mesh Recovery](https://mlsanity.com/paper/2602.15989/) (2026)
- [EgoHumans: An Egocentric 3D Multi-Human Benchmark](https://mlsanity.com/paper/2305.16487/) (2023)
- [Estimating Body and Hand Motion in an Ego-sensed World](https://mlsanity.com/paper/2410.03665/) (2024)
- [GENMO: A GENeralist Model for Human MOtion](https://mlsanity.com/paper/2505.01425/) (2025)
- [RoHM: Robust Human Motion Reconstruction via Diffusion](https://mlsanity.com/paper/2401.08570/) (2024)
- [Human3R: Everyone Everywhere All at Once](https://mlsanity.com/paper/2510.06219/) (2025)
- [EgoPoser: Robust Real-Time Egocentric Pose Estimation from Sparse and Intermittent Observations Everywhere](https://mlsanity.com/paper/2308.06493/) (2023)
- [A generic diffusion-based approach for 3D human pose prediction in the wild](https://mlsanity.com/paper/2210.05669/) (2022)
- [Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions](https://mlsanity.com/paper/2410.20294/) (2024)
- [Joint Optimization for 4D Human-Scene Reconstruction in the Wild](https://mlsanity.com/paper/2501.02158/) (2025)
- [CoMotion: Concurrent Multi-person 3D Motion](https://mlsanity.com/paper/2504.12186/) (2025)

## Abstract (arXiv)

We address the problem of 3D body pose estimation of multiple interacting people from their egocentric views with centralized coordination. Each individual wears a camera recording egocentric video and IMU data. Processing this video with VIO SLAM provides high-quality tracking of each egocentric camera through space. The first-person view from one individual provides third-person observations of other people, although these exocentric observations are sparse, intermittent, and of highly variable reliability as both cameras and subjects move. To integrate these synchronized data streams, we propose a diffusion-based approach that fuses estimates of pose based on head motion derived from egocentric camera motion with exocentric pose observations, conditioning on both observation content and reliability. Our model is trained on a mixture of single-person motion-capture data and multi-person video in order to learn rich priors for body motion trajectories and video observation reliability. Evaluation on challenging multi-person datasets suggests our fusion approach improves over motion-only and vision-only baselines in terms of both absolute and relative pose accuracy.