# In-Home Daily-Life Captioning Using Radio Signals

arXiv: https://arxiv.org/abs/2008.10966 · HTML: https://mlsanity.com/paper/2008.10966/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2020.

## Built on by (most cited first, 10 of 10)

- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [Unsupervised Learning for Human Sensing Using Radio Signals](https://mlsanity.com/paper/2207.02370/) (2022)
- [Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable Sensors](https://mlsanity.com/paper/2410.20034/) (2024)
- [Large Model for Small Data: Foundation Model for Cross-Modal RF Human Activity Recognition](https://mlsanity.com/paper/2410.19766/) (2024)
- [Non-contact Multimodal Indoor Human Monitoring Systems: A Survey](https://mlsanity.com/paper/2312.07601/) (2023)
- [Contactless Oxygen Monitoring with Gated Transformer](https://mlsanity.com/paper/2212.03357/) (2022)
- [Radio2Text: Streaming Speech Recognition Using mmWave Radio Signals](https://mlsanity.com/paper/2308.08125/) (2023)
- [MobiDiary: Autoregressive Action Captioning with Wearable Devices and Wireless Signals](https://mlsanity.com/paper/2601.08204/) (2026)
- [RF-HOI: Recognize Human-Object Interaction with Radio Frequency Signals](https://mlsanity.com/paper/2608.00289/) (2026)
- [FSTC-Encoder: Feature--Spatial--Temporal Correlation Learning for Generalizable RF Sensing](https://mlsanity.com/paper/2608.08439/) (2026)

## Abstract (arXiv)

This paper aims to caption daily life --i.e., to create a textual description of people's activities and interactions with objects in their homes. Addressing this problem requires novel methods beyond traditional video captioning, as most people would have privacy concerns about deploying cameras throughout their homes. We introduce RF-Diary, a new model for captioning daily life by analyzing the privacy-preserving radio signal in the home with the home's floormap. RF-Diary can further observe and caption people's life through walls and occlusions and in dark settings. In designing RF-Diary, we exploit the ability of radio signals to capture people's 3D dynamics, and use the floormap to help the model learn people's interactions with objects. We also use a multi-modal feature alignment training scheme that leverages existing video-based captioning datasets to improve the performance of our radio-based captioning model. Extensive experimental results demonstrate that RF-Diary generates accurate captions under visible conditions. It also sustains its good performance in dark or occluded settings, where video-based captioning approaches fail to generate meaningful captions. For more information, please visit our project webpage: http://rf-diary.csail.mit.edu