# Non-contact Multimodal Indoor Human Monitoring Systems: A Survey

arXiv: https://arxiv.org/abs/2312.07601 · HTML: https://mlsanity.com/paper/2312.07601/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 133 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Human Action Recognition from Various Data Modalities: A Review](https://mlsanity.com/paper/2012.11866/) (2020)
- [Deep Learning for Sensor-based Activity Recognition: A Survey](https://mlsanity.com/paper/1707.03502/) (2017)
- [CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer](https://mlsanity.com/paper/2209.06535/) (2022)
- [mm-Pose: Real-Time Human Skeletal Posture Estimation using mmWave Radars and CNNs](https://mlsanity.com/paper/1911.09592/) (2019)
- [Person-in-WiFi: Fine-grained Person Perception using WiFi](https://mlsanity.com/paper/1904.00276/) (2019)
- [Deep Gait Recognition: A Survey](https://mlsanity.com/paper/2102.09546/) (2021)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar](https://mlsanity.com/paper/1911.00398/) (2019)
- [mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar](https://mlsanity.com/paper/2209.05070/) (2022)
- [Making the Invisible Visible: Action Recognition Through Walls and Occlusions](https://mlsanity.com/paper/1909.09300/) (2019)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [Pose2Room: Understanding 3D Scenes from Human Activities](https://mlsanity.com/paper/2112.03030/) (2021)
- [Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directions](https://mlsanity.com/paper/2107.11508/) (2021)
- [Gimme Signals: Discriminative signal encoding for multimodal activity recognition](https://mlsanity.com/paper/2003.06156/) (2020)
- [Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors](https://mlsanity.com/paper/2012.00201/) (2020)
- [OPERAnet: A Multimodal Activity Recognition Dataset Acquired from Radio Frequency and Vision-based Sensors](https://mlsanity.com/paper/2110.04239/) (2021)
- [GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning](https://mlsanity.com/paper/2208.14326/) (2022)
- [Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluation](https://mlsanity.com/paper/2310.15676/) (2023)
- [Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition](https://mlsanity.com/paper/2005.02335/) (2020)
- [Infrared and 3D skeleton feature fusion for RGB-D action recognition](https://mlsanity.com/paper/2002.12886/) (2020)
- [In-Home Daily-Life Captioning Using Radio Signals](https://mlsanity.com/paper/2008.10966/) (2020)
- [Multimodal sensor fusion in the latent representation space](https://mlsanity.com/paper/2208.02183/) (2022)
- [Multimodal Gait Recognition for Neurodegenerative Diseases](https://mlsanity.com/paper/2101.02469/) (2021)
- [Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars](https://mlsanity.com/paper/2207.07896/) (2022)
- [Self-Supervised Multimodal Fusion Transformer for Passive Activity Recognition](https://mlsanity.com/paper/2209.03765/) (2022)
- [CZU-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and 10 wearable inertial sensors](https://mlsanity.com/paper/2202.03283/) (2022)
- [Probabilistic Radio-Visual Active Sensing for Search and Tracking](https://mlsanity.com/paper/2011.10474/) (2020)

## Built on by (most cited first, 2 of 2)

- [Spatio-Temporal 3D Point Clouds from WiFi-CSI Data via Transformer Networks](https://mlsanity.com/paper/2410.16303/) (2024)
- [WiFi2Cap: Semantic Action Captioning from Wi-Fi CSI via Limb-Level Semantic Alignment](https://mlsanity.com/paper/2603.22690/) (2026)

## Abstract (arXiv)

Indoor human monitoring systems leverage a wide range of sensors, including cameras, radio devices, and inertial measurement units, to collect extensive data from users and the environment. These sensors contribute diverse data modalities, such as video feeds from cameras, received signal strength indicators and channel state information from WiFi devices, and three-axis acceleration data from inertial measurement units. In this context, we present a comprehensive survey of multimodal approaches for indoor human monitoring systems, with a specific focus on their relevance in elderly care. Our survey primarily highlights non-contact technologies, particularly cameras and radio devices, as key components in the development of indoor human monitoring systems. Throughout this article, we explore well-established techniques for extracting features from multimodal data sources. Our exploration extends to methodologies for fusing these features and harnessing multiple modalities to improve the accuracy and robustness of machine learning models. Furthermore, we conduct comparative analysis across different data modalities in diverse human monitoring tasks and undertake a comprehensive examination of existing multimodal datasets. This extensive survey not only highlights the significance of indoor human monitoring systems but also affirms their versatile applications. In particular, we emphasize their critical role in enhancing the quality of elderly care, offering valuable insights into the development of non-contact monitoring solutions applicable to the needs of aging populations.