# Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation

arXiv: https://arxiv.org/abs/2003.14414 · HTML: https://mlsanity.com/paper/2003.14414/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 30 of 34)

- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [SimPoE: Simulated Character Control for 3D Human Pose Estimation](https://mlsanity.com/paper/2104.00683/) (2021)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [PhyRecon: Physically Plausible Neural Scene Reconstruction](https://mlsanity.com/paper/2404.16666/) (2024)
- [Non-line-of-Sight Imaging via Neural Transient Fields](https://mlsanity.com/paper/2101.00373/) (2021)
- [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023)
- [Physics-Informed Computer Vision: A Review and Perspectives](https://mlsanity.com/paper/2305.18035/) (2023)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Physical Interaction: Reconstructing Hand-object Interactions with Physics](https://mlsanity.com/paper/2209.10833/) (2022)
- [Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging](https://mlsanity.com/paper/2205.01679/) (2022)
- [COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation](https://mlsanity.com/paper/2408.16426/) (2024)
- [NLOS-NeuS: Non-line-of-sight Neural Implicit Surface](https://mlsanity.com/paper/2303.12280/) (2023)
- [A Survey on Reinforcement Learning Methods in Character Animation](https://mlsanity.com/paper/2203.04735/) (2022)
- [Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes](https://mlsanity.com/paper/2104.13807/) (2021)
- [Few-shot Non-line-of-sight Imaging with Signal-surface Collaborative Regularization](https://mlsanity.com/paper/2211.15367/) (2022)
- [Enhancing Autonomous Navigation by Imaging Hidden Objects using Single-Photon LiDAR](https://mlsanity.com/paper/2410.03555/) (2024)
- [Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision](https://mlsanity.com/paper/2109.08880/) (2021)
- [Learning to Enhance Aperture Phasor Field for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2407.18574/) (2024)
- [Occlusion Fields: An Implicit Representation for Non-Line-of-Sight Surface Reconstruction](https://mlsanity.com/paper/2203.08657/) (2022)
- [Curvature regularization for Non-line-of-sight Imaging from Under-sampled Data](https://mlsanity.com/paper/2301.00406/) (2023)
- [Acoustic-based 3D Human Pose Estimation Robust to Human Position](https://mlsanity.com/paper/2411.07165/) (2024)
- [GraSens: A Gabor Residual Anti-aliasing Sensing Framework for Action Recognition using WiFi](https://mlsanity.com/paper/2205.11945/) (2022)
- [Domain Reduction Strategy for Non Line of Sight Imaging](https://mlsanity.com/paper/2308.10269/) (2023)
- [Multi-person Physics-based Pose Estimation for Combat Sports](https://mlsanity.com/paper/2504.08175/) (2025)
- [Noise-adapted Neural Operator for Robust Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2508.09655/) (2025)
- [Shape-aware Inertial Poser: Motion Tracking for Humans with Diverse Shapes Using Sparse Inertial Sensors](https://mlsanity.com/paper/2510.17101/) (2025)
- [Shoot-Bounce-3D: Single-Shot Occlusion-Aware 3D from Lidar by Decomposing Two-Bounce Light](https://mlsanity.com/paper/2512.06080/) (2025)
- [Unified Simulation, Perception, and Generation of Human Behavior](https://mlsanity.com/paper/2204.13678/) (2022)
- [WiFi-based Spatiotemporal Human Action Perception](https://mlsanity.com/paper/2206.09867/) (2022)

## Abstract (arXiv)

We describe a method for 3D human pose estimation from transient images (i.e., a 3D spatio-temporal histogram of photons) acquired by an optical non-line-of-sight (NLOS) imaging system. Our method can perceive 3D human pose by `looking around corners' through the use of light indirectly reflected by the environment. We bring together a diverse set of technologies from NLOS imaging, human pose estimation and deep reinforcement learning to construct an end-to-end data processing pipeline that converts a raw stream of photon measurements into a full 3D human pose sequence estimate. Our contributions are the design of data representation process which includes (1) a learnable inverse point spread function (PSF) to convert raw transient images into a deep feature vector; (2) a neural humanoid control policy conditioned on the transient image feature and learned from interactions with a physics simulator; and (3) a data synthesis and augmentation strategy based on depth data that can be transferred to a real-world NLOS imaging system. Our preliminary experiments suggest that our method is able to generalize to real-world NLOS measurement to estimate physically-valid 3D human poses.