# EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera

arXiv: https://arxiv.org/abs/1908.11505 · HTML: https://mlsanity.com/paper/1908.11505/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 30 of 65)

- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs](https://mlsanity.com/paper/2112.02789/) (2021)
- [EventNeRF: Neural Radiance Fields from a Single Colour Event Camera](https://mlsanity.com/paper/2206.11896/) (2022)
- [Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks](https://mlsanity.com/paper/2302.08890/) (2023)
- [Neural Monocular 3D Human Motion Capture with Physical Awareness](https://mlsanity.com/paper/2105.01057/) (2021)
- [Recent Event Camera Innovations: A Survey](https://mlsanity.com/paper/2408.13627/) (2024)
- [EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation](https://mlsanity.com/paper/2111.12341/) (2021)
- [LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors](https://mlsanity.com/paper/2205.15410/) (2022)
- [EventHPE: Event-based 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2108.06819/) (2021)
- [NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering using RGB Cameras](https://mlsanity.com/paper/2103.07700/) (2021)
- [LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds](https://mlsanity.com/paper/2203.14698/) (2022)
- [Dual Transfer Learning for Event-based End-task Prediction via Pluggable Event to Image Translation](https://mlsanity.com/paper/2109.01801/) (2021)
- [Gravity-Aware Monocular 3D Human-Object Reconstruction](https://mlsanity.com/paper/2108.08844/) (2021)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams](https://mlsanity.com/paper/2404.08640/) (2024)
- [Line-based 6-DoF Object Pose Estimation and Tracking With an Event Camera](https://mlsanity.com/paper/2408.03225/) (2024)
- [CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions](https://mlsanity.com/paper/2303.17948/) (2023)
- [ChallenCap: Monocular 3D Capture of Challenging Human Performances using Multi-Modal References](https://mlsanity.com/paper/2103.06747/) (2021)
- [RobustFusion: Robust Volumetric Performance Reconstruction under Human-object Interactions from Monocular RGBD Stream](https://mlsanity.com/paper/2104.14837/) (2021)
- [Efficient Human Pose Estimation via 3D Event Point Cloud](https://mlsanity.com/paper/2206.04511/) (2022)
- [Differentiable Event Stream Simulator for Non-Rigid 3D Tracking](https://mlsanity.com/paper/2104.15139/) (2021)
- [LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment](https://mlsanity.com/paper/2402.17171/) (2024)
- [HybridCap: Inertia-aid Monocular Capture of Challenging Human Motions](https://mlsanity.com/paper/2203.09287/) (2022)
- [MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes](https://mlsanity.com/paper/2208.08439/) (2022)
- [EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision](https://mlsanity.com/paper/2303.02862/) (2023)
- [3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera](https://mlsanity.com/paper/2312.14157/) (2023)
- [Microsaccade-inspired Event Camera for Robotics](https://mlsanity.com/paper/2405.17769/) (2024)
- [Lifting Monocular Events to 3D Human Poses](https://mlsanity.com/paper/2104.10609/) (2021)
- [RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method](https://mlsanity.com/paper/2403.19501/) (2024)
- [MonoNHR: Monocular Neural Human Renderer](https://mlsanity.com/paper/2210.00627/) (2022)

## Abstract (arXiv)

The high frame rate is a critical requirement for capturing fast human motions. In this setting, existing markerless image-based methods are constrained by the lighting requirement, the high data bandwidth and the consequent high computation overhead. In this paper, we propose EventCap --- the first approach for 3D capturing of high-speed human motions using a single event camera. Our method combines model-based optimization and CNN-based human pose detection to capture high-frequency motion details and to reduce the drifting in the tracking. As a result, we can capture fast motions at millisecond resolution with significantly higher data efficiency than using high frame rate videos. Experiments on our new event-based fast human motion dataset demonstrate the effectiveness and accuracy of our method, as well as its robustness to challenging lighting conditions.