mlsanityData to 7 Oct 2026

EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams

arXiv 2404.08640 ↗First posted 12 Apr 2024cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

Monocular egocentric 3D human motion capture is a challenging and actively researched problem. Existing methods use synchronously operating visual sensors (e.g. RGB cameras) and often fail under low lighting and fast motions, which can be restricting in many applications involving head-mounted devices. In response to the existing limitations, this paper 1) introduces a new problem, i.e., 3D human motion capture from an egocentric monocular event camera with a fisheye lens, and 2) proposes the first approach to it called EventEgo3D (EE3D). Event streams have high temporal resolution and provide reliable cues for 3D human motion capture under high-speed human motions and rapidly changing illumination. The proposed EE3D framework is specifically tailored for learning with event streams in the LNES representation, enabling high 3D reconstruction accuracy. We also design a prototype of a mobile head-mounted device with an event camera and record a real dataset with event observations and the ground-truth 3D human poses (in addition to the synthetic dataset). Our EE3D demonstrates robustness and superior 3D accuracy compared to existing solutions across various challenging experiments while supporting real-time 3D pose update rates of 140Hz.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 12 Apr 2024.

Builds on

37 of 52 references.

References · 37

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  5. 2019Event-based Vision: A Surveycited by 1,010
  6. 2017Learning from Synthetic Humanscited by 358
  7. 2019High Speed and High Dynamic Range Video with an Event Cameracited by 344
  8. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  9. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
  10. 2019Video to Events: Recycling Video Datasets for Event Camerascited by 162
  11. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  12. 2020BlazePose: On-device Real-time Body Pose trackingcited by 112
25 more
  1. 2021Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimationcited by 101
  2. 2022Ego-Body Pose Estimation via Ego-Head Pose Estimationcited by 93
  3. 2022EventNeRF: Neural Radiance Fields from a Single Colour Event Cameracited by 92
  4. 2019Ego-Pose Estimation and Forecasting as Real-Time PD Controlcited by 87
  5. 2019xR-EgoPose: Egocentric 3D Human Pose from an HMD Cameracited by 78
  6. 2018Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Cameracited by 67
  7. 2019EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Cameracited by 65
  8. 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 58
  9. 2021Estimating Egocentric 3D Human Pose in Global Spacecited by 56
  10. 2022UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capturecited by 56
  11. 2020SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Cameracited by 49
  12. 2020EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Streamcited by 48
  13. 2021EventHPE: Event-based 3D Human Pose and Shape Estimationcited by 42
  14. 2022Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervisioncited by 33
  15. 2023Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Viewscited by 33
  16. 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
  17. 2022Efficient Human Pose Estimation via 3D Event Point Cloudcited by 25
  18. 2016EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)cited by 24
  19. 2021Differentiable Event Stream Simulator for Non-Rigid 3D Trackingcited by 23
  20. 20233D Human Pose Perception from Egocentric Stereo Videoscited by 23
  21. 2022Event-based Non-Rigid Reconstruction from Contourscited by 17
  22. 2023EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervisioncited by 17
  23. 20233D Pose Estimation of Two Interacting Hands from a Monocular Event Cameracited by 17
  24. 2023EvAC3D: From Event-based Apparent Contours to 3D Models via Continuous Visual Hullscited by 11
  25. 2023Tracking Fast by Learning Slow: An Event-based Speed Adaptive Hand Tracker Leveraging Knowledge in RGB Domaincited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 15; 2026: 10.

29 papers.

Sort:

By area:

  1. 2024Recent Event Camera Innovations: A Surveycited by 67
  2. 2025Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understandingcited by 25
  3. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  4. 2024EMHI: A Multimodal Egocentric Human Motion Dataset with HMD and Body-Worn IMUscited by 12
  5. 2025E-3DGS: Event-Based Novel View Rendering of Large-Scale Scenes Using 3D Gaussian Splattingcited by 9
  6. 2024Helios: An extremely low power event-based gesture recognition for always-on smart eyewearcited by 8
  7. 2025Bring Your Rear Cameras for Egocentric 3D Human Pose Estimationcited by 7
  8. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 7
  9. 2025EventEgo3D++: 3D Human Motion Capture from a Head-Mounted Event Cameracited by 6
  10. 2025EventEgoHands: Event-based Egocentric 3D Hand Mesh Reconstructioncited by 5
  11. 2025Human Motion Capture from Loose and Sparse Inertial Sensors with Garment-aware Diffusion Modelscited by 5
  12. 2024Dynamic EventNeRF: Reconstructing General Dynamic Scenes from Multi-view RGB and Event Streamscited by 3
17 more
  1. 2025EgoEvGesture: Gesture Recognition Based on Egocentric Event Cameracited by 3
  2. 2025From Sharp to Blur: Unsupervised Domain Adaptation for 2D Human Pose Estimation Under Extreme Motion Blur Using Event Camerascited by 3
  3. 2026EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Cameracited by 3
  4. 2025Reading in the Dark with Foveated Event Visioncited by 2
  5. 2025EgoMusic-driven Human Dance Motion Estimation with Skeleton Mambacited by 2
  6. 2026EgoEV-HandPose: Egocentric 3D Hand Pose Estimation and Gesture Recognition with Stereo Event Camerascited by 2
  7. 2025Helios 2.0: A Robust, Ultra-Low Power Gesture Recognition System Optimised for Event-Sensor based Wearablescited by 1
  8. 2025Event-based Egocentric Human Pose Estimation in Dynamic Environmentcited by 1
  9. 2025Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reachcited by 1
  10. 2026E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimationcited by 1
  11. 2026Mocap Anywhere: Towards Pairwise-Distance based Motion Capture in the Wild (for the Wild)cited by 0
  12. 2026EventNeuS: 3D Mesh Reconstruction from a Single Event Cameracited by 0
  13. 2026EgoPoseVR: Spatiotemporal Multi-Modal Reasoning for Egocentric Full-Body Pose in Virtual Realitycited by 0
  14. 2026Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentationcited by 0
  15. 2026Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchencited by 0
  16. 2026Seeing the World and the Self from Egocentric Videocited by 0
  17. 2026EventEgoHands++: Event-based Egocentric 3D Hand Mesh Reconstruction with Real Datasetcited by 0