mlsanityData to 7 Oct 2026

MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing

arXiv 2305.10345 ↗First posted 12 May 2023eess.SP cs.AI cs.CVcited by 77

Not on the map. Why?

Abstract (arXiv)

4D human perception plays an essential role in a myriad of applications, such as home automation and metaverse avatar simulation. However, existing solutions which mainly rely on cameras and wearable devices are either privacy intrusive or inconvenient to use. To address these issues, wireless sensing has emerged as a promising alternative, leveraging LiDAR, mmWave radar, and WiFi signals for device-free human sensing. In this paper, we propose MM-Fi, the first multi-modal non-intrusive 4D human dataset with 27 daily or rehabilitation action categories, to bridge the gap between wireless sensing and high-level human perception tasks. MM-Fi consists of over 320k synchronized frames of five modalities from 40 human subjects. Various annotations are provided to support potential sensing tasks, e.g., human pose estimation and action recognition. Extensive experiments have been conducted to compare the sensing capacity of each or several modalities in terms of multiple tasks. We envision that MM-Fi can contribute to wireless sensing research with respect to action recognition, human pose estimation, multi-modal learning, cross-modal supervision, and interdisciplinary healthcare research.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Builds on 31 of its 59 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

31 of 59 references.

References · 31

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2020Point Transformercited by 1,462
  3. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  4. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  5. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  6. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  7. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  8. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 317
  9. 2022HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modelingcited by 115
  10. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  11. 2021All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agendacited by 67
  12. 2019mm-Pose: Real-Time Human Skeletal Posture Estimation using mmWave Radars and CNNscited by 58
19 more
  1. 2020MoVi: A Large Multipurpose Motion and Video Datasetcited by 58
  2. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  3. 2023SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environmentscited by 47
  4. 2022LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensorscited by 43
  5. 2021Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Drivingcited by 39
  6. 2022mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensorscited by 37
  7. 2019A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentationcited by 36
  8. 2022SenseFi: A Library and Benchmark on Deep-Learning-Empowered WiFi Human Sensingcited by 34
  9. 2022LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Cloudscited by 32
  10. 2022mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radarcited by 32
  11. 2020Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approachcited by 27
  12. 2023CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactionscited by 27
  13. 2020Transferring Dense Pose to Proximal Animal Classescited by 23
  14. 2022Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDARcited by 23
  15. 2022MetaFi: Device-Free Pose Estimation via Commodity WiFi for Metaverse Avatar Simulationcited by 22
  16. 2022Fast and Scalable Human Pose Estimation using mmWave Point Cloudcited by 20
  17. 2015Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approachcited by 16
  18. 2020Deep Learning for Radio-based Human Sensing: Recent Advances and Future Directionscited by 12
  19. 2022DensePose From WiFicited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 17; 2025: 33; 2026: 26.

77 papers.

Sort:

By area:

  1. 2023TENT: Connect Language Models with IoT Sensors for Zero-Shot Activity Recognitioncited by 25
  2. 2024Diffusion Model is a Good Pose Estimator from 3D RF-Visioncited by 21
  3. 2024MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threatscited by 18
  4. 2024WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensingcited by 18
  5. 2024IoT-LLM: a framework for enhancing Large Language Model reasoning from real-world sensor datacited by 18
  6. 2024X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensingcited by 12
  7. 2025Towards Balanced Multi-Modal Learning in 3D Human Pose Estimationcited by 11
  8. 2024MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perceptioncited by 10
  9. 2025XRF V2: A Dataset for Action Summarization with Wi-Fi Signals, and IMUs in Phones, Watches, Earbuds, and Glassescited by 9
  10. 2025A Survey on Wi-Fi Sensing Generalizability: Taxonomy, Techniques, Datasets, and Future Research Prospectscited by 9
  11. 2025CSI-Bench: A Large-Scale In-the-Wild Dataset for Multi-task WiFi Sensingcited by 9
  12. 2024Babel: A Scalable Pre-trained Model for Multi-Modal Sensing via Expandable Modality Alignmentcited by 8
38 more
  1. 2025SensorQA: A Question Answering Benchmark for Daily-Life Monitoringcited by 7
  2. 2025RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-Wave Point Cloud Sequencecited by 7
  3. 2024RETR: Multi-View Radar Detection Transformer for Indoor Perceptioncited by 6
  4. 2025M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstructioncited by 6
  5. 2025Towards Robust and Realistic Human Pose Estimation via WiFi Signalscited by 5
  6. 2024Leveraging Foundation Models for Zero-Shot IoT Sensingcited by 4
  7. 2024Context-Aware Predictive Coding: A Representation Learning Framework for WiFi Sensingcited by 4
  8. 2025ConSense: Continually Sensing Human Activity with WiFi via Growing and Pickingcited by 4
  9. 2024Jacquard V2: Refining Datasets using the Human In the Loop Data Correction Methodcited by 3
  10. 2024MaskFi: Unsupervised Learning of WiFi and Vision Representations for Multimodal Human Activity Recognitioncited by 3
  11. 2025APML: Adaptive Probabilistic Matching Loss for Robust 3D Point Cloud Reconstructioncited by 3
  12. 2025Graph-based 3D Human Pose Estimation using WiFi Signalscited by 3
  13. 2025mmPred: Radar-based Human Motion Prediction in the Darkcited by 3
  14. 2024OMuSense-23: A Multimodal Dataset for Contactless Breathing Pattern Recognition and Biometric Analysiscited by 2
  15. 2024Spatio-Temporal 3D Point Clouds from WiFi-CSI Data via Transformer Networkscited by 2
  16. 2024Acoustic-based 3D Human Pose Estimation Robust to Human Positioncited by 2
  17. 2024DATTA: Domain-Adversarial Test-Time Adaptation for Cross-Domain WiFi-Based Human Activity Recognitioncited by 2
  18. 2025RAPTR: Radar-based 3D Pose Estimation using Transformercited by 2
  19. 2026Breaking Coordinate Overfitting: Geometry-Aware WiFi Sensing for Cross-Layout 3D Pose Estimationcited by 2
  20. 2025ADMN: A Layer-Wise Adaptive Multimodal Network for Dynamic Input Noise and Compute Resourcescited by 1
  21. 2025NVP-HRI: Zero Shot Natural Voice and Posture-based Human-Robot Interaction via Large Language Modelcited by 1
  22. 2025GenHPE: Generative Counterfactuals for 3D Human Pose Estimation with Radio Frequency Signalscited by 1
  23. 2025HoloLLM: Multisensory Foundation Model for Language-Grounded Human Sensing and Reasoningcited by 1
  24. 2025High-resolution efficient image generation from WiFi CSI using a pretrained latent diffusion modelcited by 1
  25. 2025DeSPITE: Exploring Contrastive Deep Skeleton-Pointcloud-IMU-Text Embeddings for Advanced Point Cloud Human Activity Understandingcited by 1
  26. 2025VST-Pose: A Velocity-Integrated Spatiotem-poral Attention Network for Human WiFi Pose Estimationcited by 1
  27. 2025Towards White-Box Deep Wireless Sensingcited by 1
  28. 2025Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communicationcited by 1
  29. 2025MMSense: Adapting Vision-based Foundation Model for Multi-task Multi-modal Wireless Sensingcited by 1
  30. 2026TinySense: Effective CSI Compression for Scalable and Accurate Wi-Fi Sensingcited by 1
  31. 2026WiFlow: A Lightweight WiFi-based Continuous Human Pose Estimation Network with Spatio-Temporal Feature Decouplingcited by 1
  32. 2026WiCompass: Oracle-driven Data Scaling for mmWave Human Pose Estimationcited by 1
  33. 2026Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasetscited by 1
  34. 2026COMPASS: Fusion-Matched Supervision for Missing-Modality Human Sensingcited by 1
  35. 2026DGHMesh: A Large-scale Dual-radar mmWave Dataset and Generalization-focused Benchmark for Human Mesh Reconstructioncited by 1
  36. 2026Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translationcited by 1
  37. 2024Semantic-Aware Representation of Multi-Modal Data for Data Ingress: A Literature Reviewcited by 0
  38. 2025WECAR: An End-Edge Collaborative Inference and Training Framework for WiFi-Based Continuous Human Activity Recognitioncited by 0

50 most cited of 77 All 77 (JSON)