# LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment

arXiv: https://arxiv.org/abs/2402.17171 · HTML: https://mlsanity.com/paper/2402.17171/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 71 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 30 of 43)

- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [PARE: Part Attention Regressor for 3D Human Body Estimation](https://mlsanity.com/paper/2104.08527/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [SoftGroup for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/2203.01509/) (2022)
- [DeepCap: Monocular Human Performance Capture Using Weak Supervision](https://mlsanity.com/paper/2003.08325/) (2020)
- [Probabilistic Modeling for Human Mesh Recovery](https://mlsanity.com/paper/2108.11944/) (2021)
- [Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensors](https://mlsanity.com/paper/2203.08528/) (2022)
- [HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling](https://mlsanity.com/paper/2204.13686/) (2022)
- [Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs](https://mlsanity.com/paper/1703.08014/) (2017)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [LiveCap: Real-time Human Performance Capture from Monocular Video](https://mlsanity.com/paper/1810.02648/) (2018)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)
- [SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds](https://mlsanity.com/paper/2004.02774/) (2020)
- [EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera](https://mlsanity.com/paper/1908.11505/) (2019)
- [MonoPerfCap: Human Performance Capture from Monocular Video](https://mlsanity.com/paper/1708.02136/) (2017)
- [Neural Descent for Visual 3D Human Pose and Shape](https://mlsanity.com/paper/2008.06910/) (2020)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception](https://mlsanity.com/paper/2109.05441/) (2021)
- [Harvesting Multiple Views for Marker-less 3D Human Pose Annotations](https://mlsanity.com/paper/1704.04793/) (2017)
- [SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environments](https://mlsanity.com/paper/2303.09095/) (2023)

## Built on by (most cited first, 20 of 20)

- [LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment](https://mlsanity.com/paper/2403.13307/) (2024)
- [X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing](https://mlsanity.com/paper/2410.10167/) (2024)
- [LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environment](https://mlsanity.com/paper/2407.09833/) (2024)
- [Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation](https://mlsanity.com/paper/2412.13454/) (2024)
- [FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments](https://mlsanity.com/paper/2411.04469/) (2024)
- [XR-MBT: Multi-modal Full Body Tracking for XR through Self-Supervision with Learned Depth Point Cloud Registration](https://mlsanity.com/paper/2411.18377/) (2024)
- [ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate](https://mlsanity.com/paper/2503.21268/) (2025)
- [Towards Practical Human Motion Prediction with LiDAR Point Clouds](https://mlsanity.com/paper/2408.08202/) (2024)
- [Breaking Coordinate Overfitting: Geometry-Aware WiFi Sensing for Cross-Layout 3D Pose Estimation](https://mlsanity.com/paper/2601.12252/) (2026)
- [ELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling](https://mlsanity.com/paper/2410.06963/) (2024)
- [HoloLLM: Multisensory Foundation Model for Language-Grounded Human Sensing and Reasoning](https://mlsanity.com/paper/2505.17645/) (2025)
- [Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasets](https://mlsanity.com/paper/2603.14507/) (2026)
- [FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Vision](https://mlsanity.com/paper/2603.19770/) (2026)
- [HorGait: A Hybrid Model for Accurate Gait Recognition in LiDAR Point Cloud Planar Projections](https://mlsanity.com/paper/2410.08454/) (2024)
- [WiFi based Human Fall and Activity Recognition using Transformer based Encoder Decoder and Graph Neural Networks](https://mlsanity.com/paper/2504.16655/) (2025)
- [3D Human Pose and Shape Estimation from LiDAR Point Clouds: A Review](https://mlsanity.com/paper/2509.12197/) (2025)
- [Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Dataset](https://mlsanity.com/paper/2512.10321/) (2025)
- [Sparkle: A Robust and Versatile Representation for Point Cloud based Human Motion Capture](https://mlsanity.com/paper/2604.00857/) (2026)
- [B'ezier Degradation Modeling for LiDAR-based Human Motion Capture](https://mlsanity.com/paper/2605.19620/) (2026)
- [Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration](https://mlsanity.com/paper/2608.02285/) (2026)

## Abstract (arXiv)

For human-centric large-scale scenes, fine-grained modeling for 3D human global pose and shape is significant for scene understanding and can benefit many real-world applications. In this paper, we present LiveHPS, a novel single-LiDAR-based approach for scene-level human pose and shape estimation without any limitation of light conditions and wearable devices. In particular, we design a distillation mechanism to mitigate the distribution-varying effect of LiDAR point clouds and exploit the temporal-spatial geometric and dynamic information existing in consecutive frames to solve the occlusion and noise disturbance. LiveHPS, with its efficient configuration and high-quality output, is well-suited for real-world applications. Moreover, we propose a huge human motion dataset, named FreeMotion, which is collected in various scenarios with diverse human poses, shapes and translations. It consists of multi-modal and multi-view acquisition data from calibrated and synchronized LiDARs, cameras, and IMUs. Extensive experiments on our new dataset and other public datasets demonstrate the SOTA performance and robustness of our approach. We will release our code and dataset soon.