mlsanityData to 14 Oct 2026

RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method

arXiv 2403.19501 ↗First posted 28 Mar 2024cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Comprehensive capturing of human motions requires both accurate captures of complex poses and precise localization of the human within scenes. Most of the HPE datasets and methods primarily rely on RGB, LiDAR, or IMU data. However, solely using these modalities or a combination of them may not be adequate for HPE, particularly for complex and fast movements. For holistic human motion understanding, we present RELI11D, a high-quality multimodal human motion dataset involves LiDAR, IMU system, RGB camera, and Event camera. It records the motions of 10 actors performing 5 sports in 7 scenes, including 3.32 hours of synchronized LiDAR point clouds, IMU measurement data, RGB videos and Event steams. Through extensive experiments, we demonstrate that the RELI11D presents considerable challenges and opportunities as it contains many rapid and complex motions that require precise location. To address the challenge of integrating different modalities, we propose LEIR, a multimodal baseline that effectively utilizes LiDAR Point Cloud, Event stream, and RGB through our cross-attention fusion strategy. We show that LEIR exhibits promising results for rapid motions and daily motions and that utilizing the characteristics of multiple modalities can indeed improve HPE performance. Both the dataset and source code will be released publicly to the research community, fostering collaboration and enabling further exploration in this field.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 79 of its 100 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

79 of 100 references.

References · 79

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  4. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  5. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  8. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  9. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  10. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  11. 2017End-to-end Recovery of Human Shape and Posecited by 885
  12. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
38 more
  1. 2020PCT: Point cloud transformercited by 732
  2. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  3. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  4. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  5. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  6. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  7. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  8. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  9. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  10. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 307
  11. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 295
  12. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  13. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  14. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 236
  15. 2021AGORA: Avatars in Geography Optimized for Regression Analysiscited by 205
  16. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 180
  17. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
  18. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
  19. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 153
  20. 2020TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Stylecited by 149
  21. 2022Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensorscited by 146
  22. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  23. 2020Combining Implicit Function Learning and Parametric Models for 3D Human Reconstructioncited by 133
  24. 2023One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformercited by 130
  25. 2022HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modelingcited by 117
  26. 2017Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUscited by 109
  27. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  28. 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
  29. 2023EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wildcited by 97
  30. 2020Contact and Human Dynamics from Monocular Videocited by 95
  31. 2018LiveCap: Real-time Human Performance Capture from Monocular Videocited by 93
  32. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
  33. 2021Time-Ordered Recent Event (TORE) Volumes for Event Camerascited by 73
  34. 2023X-Avatar: Expressive Human Avatarscited by 66
  35. 2019EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Cameracited by 65
  36. 2017MonoPerfCap: Human Performance Capture from Monocular Videocited by 63
  37. 2021Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimationcited by 62
  38. 2021S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modelingcited by 59

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 8.

16 papers.

Sort:

By area:

  1. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  2. 2025Joint Optimization for 4D Human-Scene Reconstruction in the Wildcited by 29
  3. 20253D Human Interaction Generation: A Surveycited by 7
  4. 2024Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimationcited by 5
  5. 2024Motion Capture from Inertial and Vision Sensorscited by 4
  6. 2024Learning to Generate Diverse Pedestrian Movements from Web Videos with Noisy Labelscited by 4
  7. 2026Towards Motion Turing Test: Evaluating Human-Likeness in Humanoid Robotscited by 4
  8. 2025ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinatecited by 3
  9. 2025From Sharp to Blur: Unsupervised Domain Adaptation for 2D Human Pose Estimation Under Extreme Motion Blur Using Event Camerascited by 3
  10. 2026FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Visioncited by 1
  11. 2026TETO: Tracking Events with Teacher Observation for Motion Estimation and Frame Interpolationcited by 1
  12. 2026VIMCAN: Visual-Inertial 3D Human Pose Estimation with Hybrid Mamba-Cross-Attention Networkcited by 1
4 more
  1. 2026EgoExoMoCap: Distributed Ego-Exo Human Motion Capturecited by 1
  2. 2026UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignmentcited by 0
  3. 2026MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observationscited by 0
  4. 2026Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integrationcited by 0