# Making the Invisible Visible: Action Recognition Through Walls and Occlusions

arXiv: https://arxiv.org/abs/1909.09300 · HTML: https://mlsanity.com/paper/1909.09300/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 25 of 25)

- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar](https://mlsanity.com/paper/2209.05070/) (2022)
- [WiMANS: A Benchmark Dataset for WiFi-based Multi-user Activity Sensing](https://mlsanity.com/paper/2402.09430/) (2024)
- [Unsupervised Learning for Human Sensing Using Radio Signals](https://mlsanity.com/paper/2207.02370/) (2022)
- [Seeing Through Clouds in Satellite Images](https://mlsanity.com/paper/2106.08408/) (2021)
- [Enabling Visual Recognition at Radio Frequency](https://mlsanity.com/paper/2405.19516/) (2024)
- [Tesla-Rapture: A Lightweight Gesture Recognition System from mmWave Radar Point Clouds](https://mlsanity.com/paper/2109.06448/) (2021)
- [Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable Sensors](https://mlsanity.com/paper/2410.20034/) (2024)
- [Self-Supervised WiFi-Based Activity Recognition](https://mlsanity.com/paper/2104.09072/) (2021)
- [On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes](https://mlsanity.com/paper/2410.19553/) (2024)
- [Look, Radiate, and Learn: Self-Supervised Localisation via Radio-Visual Correspondence](https://mlsanity.com/paper/2206.06424/) (2022)
- [hear-your-action: human action recognition by ultrasound active sensing](https://mlsanity.com/paper/2309.08087/) (2023)
- [Joint Temporal Pooling for Improving Skeleton-based Action Recognition](https://mlsanity.com/paper/2408.09356/) (2024)
- [Radio-Assisted Human Detection](https://mlsanity.com/paper/2112.08743/) (2021)
- [RFPose-OT: RF-Based 3D Human Pose Estimation via Optimal Transport Theory](https://mlsanity.com/paper/2301.13013/) (2022)
- [Deakin RF-Sensing: Experiments on Correlated Knowledge Distillation for Monitoring Human Postures with Radios](https://mlsanity.com/paper/2305.14829/) (2023)
- [Non-contact Multimodal Indoor Human Monitoring Systems: A Survey](https://mlsanity.com/paper/2312.07601/) (2023)
- [GraSens: A Gabor Residual Anti-aliasing Sensing Framework for Action Recognition using WiFi](https://mlsanity.com/paper/2205.11945/) (2022)
- [Feature-Supervised Action Modality Transfer](https://mlsanity.com/paper/2108.03329/) (2021)
- [WiFi-based Spatiotemporal Human Action Perception](https://mlsanity.com/paper/2206.09867/) (2022)
- [Encoder-Decoder Networks for Self-Supervised Pretraining and Downstream Signal Bandwidth Regression on Digital Antenna Arrays](https://mlsanity.com/paper/2307.03327/) (2023)
- [Towards Battery-Free Wireless Sensing via Radio-Frequency Energy Harvesting](https://mlsanity.com/paper/2409.00086/) (2024)
- [OccludeNet: A Causal Journey into Mixed-View Actor-Centric Video Action Recognition under Occlusions](https://mlsanity.com/paper/2411.15729/) (2024)
- [Affinity Contrastive Learning for Skeleton-based Human Activity Understanding](https://mlsanity.com/paper/2601.16694/) (2026)
- [FSTC-Encoder: Feature--Spatial--Temporal Correlation Learning for Generalizable RF Sensing](https://mlsanity.com/paper/2608.08439/) (2026)

## Abstract (arXiv)

Understanding people's actions and interactions typically depends on seeing them. Automating the process of action recognition from visual data has been the topic of much research in the computer vision community. But what if it is too dark, or if the person is occluded or behind a wall? In this paper, we introduce a neural network model that can detect human actions through walls and occlusions, and in poor lighting conditions. Our model takes radio frequency (RF) signals as input, generates 3D human skeletons as an intermediate representation, and recognizes actions and interactions of multiple people over time. By translating the input to an intermediate skeleton-based representation, our model can learn from both vision-based and RF-based datasets, and allow the two tasks to help each other. We show that our model achieves comparable accuracy to vision-based action recognition systems in visible scenarios, yet continues to work accurately when people are not visible, hence addressing scenarios that are beyond the limit of today's vision-based action recognition.