# Recovering Surveillance Video Using RF Cues

arXiv: https://arxiv.org/abs/2212.13340 · HTML: https://mlsanity.com/paper/2212.13340/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 27 Dec 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [Exploiting temporal information for 3D pose estimation](https://mlsanity.com/paper/1711.08585/) (2017)
- [Modality Distillation with Multiple Stream Networks for Action Recognition](https://mlsanity.com/paper/1806.07110/) (2018)
- [Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image](https://mlsanity.com/paper/1701.00295/) (2017)
- [Pose Guided Human Video Generation](https://mlsanity.com/paper/1807.11152/) (2018)
- [Person-in-WiFi: Fine-grained Person Perception using WiFi](https://mlsanity.com/paper/1904.00276/) (2019)
- [Deep Video Generation, Prediction and Completion of Human Action Sequences](https://mlsanity.com/paper/1711.08682/) (2017)
- [SOLOv2: Dynamic and Fast Instance Segmentation](https://mlsanity.com/paper/2003.10152/) (2020)
- [Hybrid Instance-aware Temporal Fusion for Online Video Instance Segmentation](https://mlsanity.com/paper/2112.01695/) (2021)
- [Video Instance Segmentation by Instance Flow Assembly](https://mlsanity.com/paper/2110.10599/) (2021)
- [Panoramic Video Salient Object Detection with Ambisonic Audio Guidance](https://mlsanity.com/paper/2211.14419/) (2022)
- [Towards Cross-Modal Forgery Detection and Localization on Live Surveillance Videos](https://mlsanity.com/paper/2101.00848/) (2021)
- [Combining LiDAR Space Clustering and Convolutional Neural Networks for Pedestrian Detection](https://mlsanity.com/paper/1710.06160/) (2017)
- [Online Video Instance Segmentation via Robust Context Fusion](https://mlsanity.com/paper/2207.05580/) (2022)
- [Forgery Attack Detection in Surveillance Video Streams Using Wi-Fi Channel State Information](https://mlsanity.com/paper/2201.09487/) (2022)
- [Self-supervised Multi-Modal Video Forgery Attack Detection](https://mlsanity.com/paper/2209.06345/) (2022)

## Abstract (arXiv)

Video capture is the most extensively utilized human perception source due to its intuitively understandable nature. A desired video capture often requires multiple environmental conditions such as ample ambient-light, unobstructed space, and proper camera angle. In contrast, wireless measurements are more ubiquitous and have fewer environmental constraints. In this paper, we propose CSI2Video, a novel cross-modal method that leverages only WiFi signals from commercial devices and a source of human identity information to recover fine-grained surveillance video in a real-time manner. Specifically, two tailored deep neural networks are designed to conduct cross-modal mapping and video generation tasks respectively. We make use of an auto-encoder-based structure to extract pose features from WiFi frames. Afterward, both extracted pose features and identity information are merged to generate synthetic surveillance video. Our solution generates realistic surveillance videos without any expensive wireless equipment and has ubiquitous, cheap, and real-time characteristics.