mlsanityData to 7 Oct 2026

mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors

arXiv 2210.08394 ↗First posted 15 Oct 2022cs.CVcited by 37

Not on the map. Why?

Abstract (arXiv)

The ability to estimate 3D human body pose and movement, also known as human pose estimation (HPE), enables many applications for home-based health monitoring, such as remote rehabilitation training. Several possible solutions have emerged using sensors ranging from RGB cameras, depth sensors, millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite previous efforts on datasets and benchmarks for HPE, few dataset exploits multiple modalities and focuses on home-based health monitoring. To bridge the gap, we present mRI, a multi-modal 3D human pose estimation dataset with mmWave, RGB-D, and Inertial Sensors. Our dataset consists of over 160k synchronized frames from 20 subjects performing rehabilitation exercises and supports the benchmarks of HPE and action detection. We perform extensive experiments using our dataset and delineate the strength of each modality. We hope that the release of mRI can catalyze the research in pose estimation, multi-modal learning, and action understanding, and more importantly facilitate the applications of home-based health monitoring.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 14 of its 56 references that are in the map. First posted on arXiv 15 Oct 2022.

Builds on

14 of 56 references.

References · 14

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017Mask R-CNNcited by 7,259
  3. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  4. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  5. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  6. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  7. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  8. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  9. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  10. 2017Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUscited by 108
  11. 2019mm-Pose: Real-Time Human Skeletal Posture Estimation using mmWave Radars and CNNscited by 58
  12. 2020MoVi: A Large Multipurpose Motion and Video Datasetcited by 58
2 more
  1. 2022Fast and Scalable Human Pose Estimation using mmWave Point Cloudcited by 20
  2. 2020Transfer Learning for Human Activity Recognition using Representational Analysis of Neural Networkscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 10; 2025: 7; 2026: 13.

37 papers.

Sort:

By area:

  1. 2023MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensingcited by 77
  2. 2023Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensorscited by 29
  3. 2023TENT: Connect Language Models with IoT Sensors for Zero-Shot Activity Recognitioncited by 25
  4. 2024Diffusion Model is a Good Pose Estimator from 3D RF-Visioncited by 21
  5. 2024RELI11D: A Comprehensive Multimodal Human Motion Dataset and Methodcited by 16
  6. 2024RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmarkcited by 16
  7. 2023MiliPoint: A Point Cloud Dataset for mmWave Radarcited by 13
  8. 2024X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensingcited by 12
  9. 2024MESEN: Exploit Multimodal Data to Design Unimodal Human Activity Recognition with Few Labelscited by 11
  10. 2024MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perceptioncited by 10
  11. 2024Babel: A Scalable Pre-trained Model for Multi-Modal Sensing via Expandable Modality Alignmentcited by 8
  12. 2023MMBaT: A Multi-task Framework for mmWave-based Human Body Reconstruction and Translation Predictioncited by 7
25 more
  1. 2025RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-Wave Point Cloud Sequencecited by 7
  2. 2024RETR: Multi-View Radar Detection Transformer for Indoor Perceptioncited by 6
  3. 2025M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstructioncited by 6
  4. 2025mmDEAR: mmWave Point Cloud Density Enhancement for Accurate Human Body Reconstructioncited by 5
  5. 2024ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusioncited by 4
  6. 2024Acoustic-based 3D Human Pose Estimation Robust to Human Positioncited by 2
  7. 2025RAPTR: Radar-based 3D Pose Estimation using Transformercited by 2
  8. 2023Intelligent Knee Sleeves: A Real-time Multimodal Dataset for 3D Lower Body Motion Estimation Using Smart Textilecited by 1
  9. 2025GenHPE: Generative Counterfactuals for 3D Human Pose Estimation with Radio Frequency Signalscited by 1
  10. 2025mmJoints: Expanding Joint Representations Beyond (x,y,z) in mmWave-Based 3D Pose Estimationcited by 1
  11. 2025mmWEAVER: Environment-Specific mmWave Signal Synthesis from a Photo and Activity Descriptioncited by 1
  12. 2026WiCompass: Oracle-driven Data Scaling for mmWave Human Pose Estimationcited by 1
  13. 2026MoBind: Motion Binding for Fine-Grained IMU-Video Pose Alignmentcited by 1
  14. 2026Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasetscited by 1
  15. 2026FlashCap: Millisecond-Accurate Human Motion Capture via Flashing LEDs and Event-Based Visioncited by 1
  16. 2026DGHMesh: A Large-scale Dual-radar mmWave Dataset and Generalization-focused Benchmark for Human Mesh Reconstructioncited by 1
  17. 2026Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translationcited by 1
  18. 20233D Pose Nowcasting: Forecast the Future to Improve the Presentcited by 0
  19. 2026PCFEx: Point Cloud Feature Extraction for Graph Neural Networkscited by 0
  20. 2026mmGAT: Pose Estimation by Graph Attention with Mutual Features from mmWave Radar Point Cloudcited by 0
  21. 2026UMEDA: Unified Multi-modal Efficient Data Fusion for Privacy-Preserving Graph Federated Learning via Spectral-Gated Attention and Diffusion-Based Operator Alignmentcited by 0
  22. 2026DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognitioncited by 0
  23. 2026Learning Biomechanically Plausible Human Motion from Sparse Radar Point Cloudscited by 0
  24. 2026You Only Flow Once: Calibrated and Real-Time Radar Pose Estimation with Multi-Hypothesis Normalizing Flowscited by 0
  25. 2026Can Language Models Understand mmWave Data? Benchmarking Large Language Models for mmWave Radar-Based Human Understandingcited by 0