# EventHPE: Event-based 3D Human Pose and Shape Estimation

arXiv: https://arxiv.org/abs/2108.06819 · HTML: https://mlsanity.com/paper/2108.06819/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 36 references that are in the map.
- First posted on arXiv 15 Aug 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras](https://mlsanity.com/paper/1802.06898/) (2018)
- [End-to-End Learning of Representations for Asynchronous Event-Based Data](https://mlsanity.com/paper/1904.08245/) (2019)
- [Asynchronous, Photometric Feature Tracking using Events and Frames](https://mlsanity.com/paper/1807.09713/) (2018)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare](https://mlsanity.com/paper/1910.00116/) (2019)
- [Learning Event-Based Motion Deblurring](https://mlsanity.com/paper/2004.05794/) (2020)
- [EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera](https://mlsanity.com/paper/1908.11505/) (2019)
- [Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/1511.09439/) (2015)
- [Event-based, 6-DOF Camera Tracking from Photometric Depth Maps](https://mlsanity.com/paper/1607.03468/) (2016)
- [A Progressive Batching L-BFGS Method for Machine Learning](https://mlsanity.com/paper/1802.05374/) (2018)
- [Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimation](https://mlsanity.com/paper/1508.06708/) (2015)
- [3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information](https://mlsanity.com/paper/1608.03075/) (2016)
- [Robust Estimation of 3D Human Poses from a Single Image](https://mlsanity.com/paper/1406.2282/) (2014)
- [3D Human Pose Machines with Self-supervised Learning](https://mlsanity.com/paper/1901.03798/) (2019)
- [SparseFusion: Dynamic Human Avatar Modeling from Sparse RGBD Images](https://mlsanity.com/paper/2006.03630/) (2020)
- [Detailed Avatar Recovery from Single Image](https://mlsanity.com/paper/2108.02931/) (2021)
- [A Simple, Fast and Highly-Accurate Algorithm to Recover 3D Shape from 2D Landmarks on a Single Image](https://mlsanity.com/paper/1609.09058/) (2016)

## Built on by (most cited first, 30 of 42)

- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams](https://mlsanity.com/paper/2404.08640/) (2024)
- [Efficient Human Pose Estimation via 3D Event Point Cloud](https://mlsanity.com/paper/2206.04511/) (2022)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision](https://mlsanity.com/paper/2303.02862/) (2023)
- [Microsaccade-inspired Event Camera for Robotics](https://mlsanity.com/paper/2405.17769/) (2024)
- [RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method](https://mlsanity.com/paper/2403.19501/) (2024)
- [Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction](https://mlsanity.com/paper/2403.07346/) (2024)
- [A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception](https://mlsanity.com/paper/2303.14176/) (2023)
- [Highly Efficient 3D Human Pose Tracking from Events with Spiking Spatiotemporal Transformer](https://mlsanity.com/paper/2303.09681/) (2023)
- [Neuromorphic High-Frequency 3D Dancing Pose Estimation in Dynamic Environment](https://mlsanity.com/paper/2301.06648/) (2023)
- [Lightweight Super-Resolution Head for Human Pose Estimation](https://mlsanity.com/paper/2307.16765/) (2023)
- [E-4DGS: High-Fidelity Dynamic Reconstruction from the Multi-view Event Cameras](https://mlsanity.com/paper/2508.09912/) (2025)
- [GTPT: Group-based Token Pruning Transformer for Efficient Human Pose Estimation](https://mlsanity.com/paper/2407.10756/) (2024)
- [EventEgo3D++: 3D Human Motion Capture from a Head-Mounted Event Camera](https://mlsanity.com/paper/2502.07869/) (2025)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [A Temporal Densely Connected Recurrent Network for Event-based Human Pose Estimation](https://mlsanity.com/paper/2209.07034/) (2022)
- [EventEgoHands: Event-based Egocentric 3D Hand Mesh Reconstruction](https://mlsanity.com/paper/2505.19169/) (2025)
- [Exploring Event-based Human Pose Estimation with 3D Event Representations](https://mlsanity.com/paper/2311.04591/) (2023)
- [Dynamic EventNeRF: Reconstructing General Dynamic Scenes from Multi-view RGB and Event Streams](https://mlsanity.com/paper/2412.06770/) (2024)
- [EgoEvGesture: Gesture Recognition Based on Egocentric Event Camera](https://mlsanity.com/paper/2503.12419/) (2025)
- [SpikeVideoFormer: An Efficient Spike-Driven Video Transformer with Hamming Attention and O(T) Complexity](https://mlsanity.com/paper/2505.10352/) (2025)
- [Hardware, Algorithms, and Applications of the Neuromorphic Vision Sensor: a Review](https://mlsanity.com/paper/2504.08588/) (2025)
- [From Sharp to Blur: Unsupervised Domain Adaptation for 2D Human Pose Estimation Under Extreme Motion Blur Using Event Cameras](https://mlsanity.com/paper/2507.22438/) (2025)
- [Sparse and Privacy-enhanced Representation for Human Pose Estimation](https://mlsanity.com/paper/2309.09515/) (2023)
- [3D Human Scan With A Moving Event Camera](https://mlsanity.com/paper/2404.08504/) (2024)
- [Exploiting Spatiotemporal Properties for Efficient Event-Driven Human Pose Estimation](https://mlsanity.com/paper/2512.06306/) (2025)

## Abstract (arXiv)

Event camera is an emerging imaging sensor for capturing dynamics of moving objects as events, which motivates our work in estimating 3D human pose and shape from the event signals. Events, on the other hand, have their unique challenges: rather than capturing static body postures, the event signals are best at capturing local motions. This leads us to propose a two-stage deep learning approach, called EventHPE. The first-stage, FlowNet, is trained by unsupervised learning to infer optical flow from events. Both events and optical flow are closely related to human body dynamics, which are fed as input to the ShapeNet in the second stage, to estimate 3D human shapes. To mitigate the discrepancy between image-based flow (optical flow) and shape-based flow (vertices movement of human body shape), a novel flow coherence loss is introduced by exploiting the fact that both flows are originated from the identical human motion. An in-house event-based 3D human dataset is curated that comes with 3D pose and shape annotations, which is by far the largest one to our knowledge. Empirical evaluations on DHP19 dataset and our in-house dataset demonstrate the effectiveness of our approach.