mlsanityData to 7 Oct 2026

EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors

arXiv 2305.01599 ↗First posted 2 May 2023cs.CV cs.GRcited by 48

Not on the map. Why?

Abstract (arXiv)

Human and environment sensing are two important topics in Computer Vision and Graphics. Human motion is often captured by inertial sensors, while the environment is mostly reconstructed using cameras. We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera. On one hand, inertial mocap suffers from large translation drift due to the lack of the global positioning signal. EgoLocate leverages image-based simultaneous localization and mapping (SLAM) techniques to locate the human in the reconstructed scene. On the other hand, SLAM often fails when the visual feature is poor. EgoLocate involves inertial mocap to provide a strong prior for the camera motion. Experiments show that localization, a key challenge for both two fields, is largely improved by our technique, compared with the state of the art of the two fields. Our codes are available for research at https://xinyu-yi.github.io/EgoLocate/.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 37 of its 71 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

37 of 71 references.

References · 37

  1. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  2. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  3. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  4. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  5. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  6. 2016Direct Sparse Odometrycited by 479
  7. 2017VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimatorcited by 418
  8. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  9. 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 151
  10. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  11. 2022Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensorscited by 144
  12. 2021Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensorscited by 137
25 more
  1. 2017Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUscited by 108
  2. 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 107
  3. 2021Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimationcited by 101
  4. 2022Ego-Body Pose Estimation via Ego-Head Pose Estimationcited by 93
  5. 2019Ego-Pose Estimation and Forecasting as Real-Time PD Controlcited by 87
  6. 2019Fast and Robust Multi-Person 3D Pose Estimation from Multiple Viewscited by 83
  7. 2019xR-EgoPose: Egocentric 3D Human Pose from an HMD Cameracited by 78
  8. 2022Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generationcited by 75
  9. 2022QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatarscited by 74
  10. 2018Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Cameracited by 67
  11. 2021Estimating Egocentric 3D Human Pose in Global Spacecited by 56
  12. 2022UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capturecited by 56
  13. 20204D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Camerascited by 49
  14. 2020SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Cameracited by 49
  15. 2016Seeing Invisible Poses: Estimating 3D Body Pose from Egocentric Videocited by 45
  16. 20204D Human Body Capture from Egocentric Video via 3D Scene Groundingcited by 43
  17. 2022DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Camerascited by 43
  18. 2021TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereocited by 39
  19. 2022HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDARcited by 38
  20. 2020Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPScited by 36
  21. 2018Direct Sparse Visual-Inertial Odometry using Dynamic Marginalizationcited by 34
  22. 2016Visual-Inertial Monocular SLAM with Map Reusecited by 32
  23. 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
  24. 2019Direct Sparse Mappingcited by 25
  25. 2022BodySLAM: Joint Camera Localisation, Mapping, and Human Motion Trackingcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 19; 2025: 19; 2026: 5.

48 papers.

Sort:

By area:

  1. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
  2. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 56
  3. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 42
  4. 2023EgoPoser: Robust Real-Time Egocentric Pose Estimation from Sparse and Intermittent Observations Everywherecited by 31
  5. 2024Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Cameracited by 30
  6. 2023DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusioncited by 29
  7. 2024EgoLM: Multi-Modal Language Model of Egocentric Motionscited by 28
  8. 2024HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Devicecited by 26
  9. 2024HMD-Poser: On-Device Real-time Human Motion Tracking from Scalable Sparse Observationscited by 22
  10. 2023SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Datacited by 20
  11. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  12. 2025MobilePoser: Real-Time Full-Body Pose Estimation and 3D Human Translation from IMUs in Mobile Consumer Devicescited by 20
36 more
  1. 2024Ultra Inertial Poser: Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Rangingcited by 18
  2. 2023Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capturecited by 17
  3. 2025Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Inputcited by 17
  4. 2024MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensorscited by 16
  5. 2024RELI11D: A Comprehensive Multimodal Human Motion Dataset and Methodcited by 16
  6. 2024A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signalscited by 15
  7. 2025EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activitycited by 15
  8. 2024DivaTrack: Diverse Bodies and Motions from Acceleration-Enhanced Three-Point Trackerscited by 11
  9. 2024EgoHDM: An Online Egocentric-Inertial Human Motion Capture, Localization, and Dense Mapping Systemcited by 9
  10. 2025ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videoscited by 8
  11. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8
  12. 2025Bring Your Rear Cameras for Egocentric 3D Human Pose Estimationcited by 7
  13. 2025EventEgo3D++: 3D Human Motion Capture from a Head-Mounted Event Cameracited by 6
  14. 2024HiSC4D: Human-centered interaction and 4D Scene Capture in Large-scale Space Using Wearable IMUs and LiDARcited by 5
  15. 2025FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Videocited by 5
  16. 2025UMotion: Uncertainty-driven Human Motion Estimation from Inertial and Ultra-wideband Unitscited by 5
  17. 2025Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokenscited by 4
  18. 2026RAG-GFM: Overcoming In-Memory Bottlenecks in Graph Foundation Models via Retrieval-Augmented Generationcited by 4
  19. 2024DragPoser: Motion Reconstruction from Variable Sparse Tracking Signals via Latent Space Optimizationcited by 3
  20. 2024Improving Neural Surface Reconstruction with Feature Priors from Multi-View Imagecited by 3
  21. 2025Group Inertial Poser: Multi-Person Pose and Global Translation from Sparse Inertial Sensors and Ultra-Wideband Rangingcited by 3
  22. 2024Self-Avatar Animation in Virtual Reality: Impact of Motion Signals Artifacts on the Full-Body Pose Reconstructioncited by 2
  23. 2024VIPeR: Visual Incremental Place Recognition with Adaptive Mining and Continual Learningcited by 2
  24. 2025Egocentric Action-aware Inertial Localization in Point Clouds with Vision-Language Guidancecited by 2
  25. 2025DiffCap: Diffusion-based Real-time Human Motion Capture using Sparse IMUs and a Monocular Cameracited by 2
  26. 2026Ground Reaction Inertial Poser: Physics-based Human Motion Capture from Sparse IMUs and Insole Pressure Sensorscited by 2
  27. 2025SSPINNpose: A Self-Supervised PINN for Inertial Pose and Dynamics Estimationcited by 1
  28. 2025YOWO: You Only Walk Once to Jointly Map An Indoor Scene and Register Ceiling-mounted Camerascited by 1
  29. 2024A Mixture of Experts Approach to 3D Human Motion Predictioncited by 0
  30. 2024Social EgoMesh Estimationcited by 0
  31. 2025Step2Motion: Locomotion Reconstruction from Pressure Sensing Insolescited by 0
  32. 2025OmniEgoCap: Camera-Agnostic Sequence-Level Egocentric Motion Reconstructioncited by 0
  33. 2025Human Motion Estimation with Everyday Wearablescited by 0
  34. 2026Mocap Anywhere: Towards Pairwise-Distance based Motion Capture in the Wild (for the Wild)cited by 0
  35. 2026MotionGRPO: Overcoming Low Intra-Group Diversity in GRPO-Based Egocentric Motion Recoverycited by 0
  36. 2026AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalizationcited by 0