# XRF V2: A Dataset for Action Summarization with Wi-Fi Signals, and IMUs in Phones, Watches, Earbuds, and Glasses

arXiv: https://arxiv.org/abs/2501.19034 · HTML: https://mlsanity.com/paper/2501.19034/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 93 references that are in the map.
- First posted on arXiv 31 Jan 2025.

## Builds on (most cited first, 24 of 24)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding](https://mlsanity.com/paper/2403.09626/) (2024)
- [MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing](https://mlsanity.com/paper/2305.10345/) (2023)
- [Person-in-WiFi: Fine-grained Person Perception using WiFi](https://mlsanity.com/paper/1904.00276/) (2019)
- [IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds](https://mlsanity.com/paper/2304.12518/) (2023)
- [Can WiFi Estimate Person Pose?](https://mlsanity.com/paper/1904.00277/) (2019)
- [Joint Activity Recognition and Indoor Localization with WiFi Fingerprints](https://mlsanity.com/paper/1904.04964/) (2019)
- [Hang-Time HAR: A Benchmark Dataset for Basketball Activity Recognition using Wrist-Worn Inertial Sensors](https://mlsanity.com/paper/2305.13124/) (2023)
- [Interaction Methods for Smart Glasses](https://mlsanity.com/paper/1707.09728/) (2017)
- [Temporal Action Localization for Inertial-based Human Activity Recognition](https://mlsanity.com/paper/2311.15831/) (2023)
- [WiFi CSI Based Temporal Activity Detection via Dual Pyramid Network](https://mlsanity.com/paper/2412.16233/) (2024)
- [You Can Wash Hands Better: Accurate Daily Handwashing Assessment with a Smartwatch](https://mlsanity.com/paper/2112.06657/) (2021)
- [TensorBeat: Tensor Decomposition for Monitoring Multi-Person Breathing Beats with Commodity WiFi](https://mlsanity.com/paper/1702.02046/) (2017)
- [Mask Wearing Status Estimation with Smartwatches](https://mlsanity.com/paper/2205.06113/) (2022)

## Built on by (most cited first, 9 of 9)

- [A Survey on Wi-Fi Sensing Generalizability: Taxonomy, Techniques, Datasets, and Future Research Prospects](https://mlsanity.com/paper/2503.08008/) (2025)
- [Talk is Not Always Cheap: Promoting Wireless Sensing Models with Text Prompts](https://mlsanity.com/paper/2504.14621/) (2025)
- [Cross-Domain Multi-Person Human Activity Recognition via Near-Field Wi-Fi Sensing](https://mlsanity.com/paper/2510.17816/) (2025)
- [Scale What Counts, Mask What Matters: Evaluating Foundation Models for Zero-Shot Cross-Domain Wi-Fi Sensing](https://mlsanity.com/paper/2511.18792/) (2025)
- [MobiDiary: Autoregressive Action Captioning with Wearable Devices and Wireless Signals](https://mlsanity.com/paper/2601.08204/) (2026)
- [WS-IMUBench: Can Weakly Supervised Methods from Audio, Image, and Video Be Adapted for IMU-based Temporal Action Localization?](https://mlsanity.com/paper/2602.01850/) (2026)
- [Purify-then-Align: Towards Robust Human Sensing under Modality Missing with Knowledge Distillation from Noisy Multimodal Teacher](https://mlsanity.com/paper/2604.05584/) (2026)
- [RF-HOI: Recognize Human-Object Interaction with Radio Frequency Signals](https://mlsanity.com/paper/2608.00289/) (2026)
- [PSEE: Progressive Sensor Event Expansion for Point-Supervised Temporal Action Localization](https://mlsanity.com/paper/2609.21462/) (2026)

## Abstract (arXiv)

Human Action Recognition (HAR) plays a crucial role in applications such as health monitoring, smart home automation, and human-computer interaction. While HAR has been extensively studied, action summarization using Wi-Fi and IMU signals in smart-home environments , which involves identifying and summarizing continuous actions, remains an emerging task. This paper introduces the novel XRF V2 dataset, designed for indoor daily activity Temporal Action Localization (TAL) and action summarization. XRF V2 integrates multimodal data from Wi-Fi signals, IMU sensors (smartphones, smartwatches, headphones, and smart glasses), and synchronized video recordings, offering a diverse collection of indoor activities from 16 volunteers across three distinct environments. To tackle TAL and action summarization, we propose the XRFMamba neural network, which excels at capturing long-term dependencies in untrimmed sensory sequences and achieves the best performance with an average mAP of 78.74, outperforming the recent WiFiTAD by 5.49 points in mAP@avg while using 35% fewer parameters. In action summarization, we introduce a new metric, Response Meaning Consistency (RMC), to evaluate action summarization performance. And it achieves an average Response Meaning Consistency (mRMC) of 0.802. We envision XRF V2 as a valuable resource for advancing research in human action localization, action forecasting, pose estimation, multimodal foundation models pre-training, synthetic data generation, and more. The data and code are available at https://github.com/aiotgroup/XRFV2.