# LPSNet: End-to-End Human Pose and Shape Estimation with Lensless Imaging

arXiv: https://arxiv.org/abs/2404.01941 · HTML: https://mlsanity.com/paper/2404.01941/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 24 of 24)

- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Humans in 4D: Reconstructing and Tracking Humans with Transformers](https://mlsanity.com/paper/2305.20091/) (2023)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop](https://mlsanity.com/paper/2103.16507/) (2021)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [Exploiting temporal context for 3D human pose estimation in the wild](https://mlsanity.com/paper/1905.04266/) (2019)
- [Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields](https://mlsanity.com/paper/2207.13807/) (2022)
- [Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers](https://mlsanity.com/paper/2207.13820/) (2022)
- [Hierarchical Kinematic Human Mesh Recovery](https://mlsanity.com/paper/2003.04232/) (2020)
- [SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation](https://mlsanity.com/paper/2107.03332/) (2021)
- [Human Mesh Recovery from Multiple Shots](https://mlsanity.com/paper/2012.09843/) (2020)
- [DiffuserCam: Lensless Single-exposure 3D Imaging](https://mlsanity.com/paper/1710.02134/) (2017)
- [Predicting 3D Human Dynamics from Video](https://mlsanity.com/paper/1908.04781/) (2019)
- [Learning 3D Human Shape and Pose from Dense Body Parts](https://mlsanity.com/paper/1912.13344/) (2019)
- [SMPLy Benchmarking 3D Human Pose Estimation in the Wild](https://mlsanity.com/paper/2012.02743/) (2020)

## Built on by (most cited first, 3 of 3)

- [GANESH: Generalizable NeRF for Lensless Imaging](https://mlsanity.com/paper/2411.04810/) (2024)
- [On the Use of Hierarchical Vision Foundation Models for Low-Cost Human Mesh Recovery and Pose Estimation](https://mlsanity.com/paper/2510.12660/) (2025)
- [Integrated Forward-Inverse Network for Lensless Image Reconstruction](https://mlsanity.com/paper/2607.04608/) (2026)

## Abstract (arXiv)

Human pose and shape (HPS) estimation with lensless imaging is not only beneficial to privacy protection but also can be used in covert surveillance scenarios due to the small size and simple structure of this device. However, this task presents significant challenges due to the inherent ambiguity of the captured measurements and lacks effective methods for directly estimating human pose and shape from lensless data. In this paper, we propose the first end-to-end framework to recover 3D human poses and shapes from lensless measurements to our knowledge. We specifically design a multi-scale lensless feature decoder to decode the lensless measurements through the optically encoded mask for efficient feature extraction. We also propose a double-head auxiliary supervision mechanism to improve the estimation accuracy of human limb ends. Besides, we establish a lensless imaging system and verify the effectiveness of our method on various datasets acquired by our lensless imaging system.