# Real-time, low-cost multi-person 3D pose estimation

arXiv: https://arxiv.org/abs/2110.11414 · HTML: https://mlsanity.com/paper/2110.11414/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 60 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/1711.07399/) (2017)
- [Feature Boosting Network For 3D Pose Estimation](https://mlsanity.com/paper/1901.04877/) (2019)
- [Neural network identification of people hidden from view with a single-pixel, single-photon detector](https://mlsanity.com/paper/1709.07244/) (2017)
- [Spatial images from temporal data](https://mlsanity.com/paper/1912.01413/) (2019)
- [Robust super-resolution depth imaging via a multi-feature fusion deep network](https://mlsanity.com/paper/2011.11444/) (2020)
- [Learning Human Pose Models from Synthesized Data for Robust RGB-D Action Recognition](https://mlsanity.com/paper/1707.00823/) (2017)
- [Toward Marker-free 3D Pose Estimation in Lifting: A Deep Multi-view Solution](https://mlsanity.com/paper/1802.01741/) (2018)
- [3D imaging from multipath temporal echoes](https://mlsanity.com/paper/2011.09284/) (2020)
- [Upper Body Pose Estimation Using Wearable Inertial Sensors and Multiplicative Kalman Filter](https://mlsanity.com/paper/1909.10376/) (2019)

## Built on by (most cited first, 2 of 2)

- [Unlocking the Performance of Proximity Sensors by Utilizing Transient Histograms](https://mlsanity.com/paper/2308.13473/) (2023)
- [Simulating single-photon detector array sensors for depth imaging](https://mlsanity.com/paper/2210.05644/) (2022)

## Abstract (arXiv)

The process of tracking human anatomy in computer vision is referred to pose estimation, and it is used in fields ranging from gaming to surveillance. Three-dimensional pose estimation traditionally requires advanced equipment, such as multiple linked intensity cameras or high-resolution time-of-flight cameras to produce depth images. However, there are applications, e.g.~consumer electronics, where significant constraints are placed on the size, power consumption, weight and cost of the usable technology. Here, we demonstrate that computational imaging methods can achieve accurate pose estimation and overcome the apparent limitations of time-of-flight sensors designed for much simpler tasks. The sensor we use is already widely integrated in consumer-grade mobile devices, and despite its low spatial resolution, only 4×4 pixels, our proposed Pixels2Pose system transforms its data into accurate depth maps and 3D pose data of multiple people up to a distance of 3 m from the sensor. We are able to generate depth maps at a resolution of 32×32 and 3D localization of a body parts with an error of only ≈10 cm at a frame rate of 7 fps. This work opens up promising real-life applications in scenarios that were previously restricted by the advanced hardware requirements and cost of time-of-flight technology.