# Bring Your Rear Cameras for Egocentric 3D Human Pose Estimation

arXiv: https://arxiv.org/abs/2503.11652 · HTML: https://mlsanity.com/paper/2503.11652/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 67 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perception](https://mlsanity.com/paper/2306.06362/) (2023)
- [MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video](https://mlsanity.com/paper/2203.00859/) (2022)
- [MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2111.12707/) (2021)
- [EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices](https://mlsanity.com/paper/2112.07642/) (2021)
- [Epipolar Transformers](https://mlsanity.com/paper/2005.04551/) (2020)
- [AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing](https://mlsanity.com/paper/2207.13784/) (2022)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.14304/) (2021)
- [Ego-Body Pose Estimation via Ego-Head Pose Estimation](https://mlsanity.com/paper/2212.04636/) (2022)
- [PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2303.17472/) (2023)
- [Ego-Pose Estimation and Forecasting as Real-Time PD Control](https://mlsanity.com/paper/1906.03173/) (2019)
- [xR-EgoPose: Egocentric 3D Human Pose from an HMD Camera](https://mlsanity.com/paper/1907.10045/) (2019)
- [Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera](https://mlsanity.com/paper/1803.05959/) (2018)
- [EgoHumans: An Egocentric 3D Multi-Human Benchmark](https://mlsanity.com/paper/2305.16487/) (2023)
- [UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture](https://mlsanity.com/paper/2208.01633/) (2022)
- [Estimating Egocentric 3D Human Pose in Global Space](https://mlsanity.com/paper/2104.13454/) (2021)
- [SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera](https://mlsanity.com/paper/2011.01519/) (2020)
- [EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors](https://mlsanity.com/paper/2305.01599/) (2023)
- [Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision](https://mlsanity.com/paper/2201.07929/) (2022)
- [EgoPoser: Robust Real-Time Egocentric Pose Estimation from Sparse and Intermittent Observations Everywhere](https://mlsanity.com/paper/2308.06493/) (2023)
- [Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers](https://mlsanity.com/paper/2210.06110/) (2022)

## Built on by (most cited first, 7 of 7)

- [EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR](https://mlsanity.com/paper/2603.04090/) (2026)
- [Egocentric Visibility-Aware Human Pose Estimation](https://mlsanity.com/paper/2602.23618/) (2026)
- [E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation](https://mlsanity.com/paper/2604.08543/) (2026)
- [OmniEgoCap: Camera-Agnostic Sequence-Level Egocentric Motion Reconstruction](https://mlsanity.com/paper/2512.19283/) (2025)
- [Human Motion Estimation with Everyday Wearables](https://mlsanity.com/paper/2512.21209/) (2025)
- [EgoPoseVR: Spatiotemporal Multi-Modal Reasoning for Egocentric Full-Body Pose in Virtual Reality](https://mlsanity.com/paper/2602.05590/) (2026)
- [Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting](https://mlsanity.com/paper/2607.15890/) (2026)

## Abstract (arXiv)

Egocentric 3D human pose estimation has been actively studied using cameras installed in front of a head-mounted device (HMD). While frontal placement is the optimal and the only option for some tasks, such as hand tracking, it remains unclear if the same holds for full-body tracking due to self-occlusion and limited field-of-view coverage. Notably, even the state-of-the-art methods often fail to estimate accurate 3D poses in many scenarios, such as when HMD users tilt their heads upward -- a common motion in human activities. A key limitation of existing HMD designs is their neglect of the back of the body, despite its potential to provide crucial 3D reconstruction cues. Hence, this paper investigates the usefulness of rear cameras for full-body tracking. We also show that simply adding rear views to the frontal inputs is not optimal for existing methods due to their dependence on individual 2D joint detectors without effective multi-view integration. To address this issue, we propose a new transformer-based method that refines 2D joint heatmap estimation with multi-view information and heatmap uncertainty, thereby improving 3D pose tracking. Also, we introduce two new large-scale datasets, Ego4View-Syn and Ego4View-RW, for a rear-view evaluation. Our experiments show that the new camera configurations with back views provide superior support for 3D pose tracking compared to only frontal placements. The proposed method achieves significant improvement over the current state of the art (>10% on MPJPE). The source code, trained models, and datasets are available on our project page at https://4dqv.mpi-inf.mpg.de/EgoRear/.