# Finding the Missing Data: A BERT-inspired Approach Against Package Loss in Wireless Sensing

arXiv: https://arxiv.org/abs/2403.12400 · HTML: https://mlsanity.com/paper/2403.12400/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Pretraining for Sequence to Sequence Learning](https://mlsanity.com/paper/1611.02683/) (2016)
- [Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI](https://mlsanity.com/paper/2207.00969/) (2022)
- [WiGest: A Ubiquitous WiFi-based Gesture Recognition System](https://mlsanity.com/paper/1501.04301/) (2015)
- [Toward Ambient Intelligence: Federated Edge Learning with Task-Oriented Sensing, Computation, and Communication Integration](https://mlsanity.com/paper/2206.05949/) (2022)
- [Integrated Sensing, Communication, and Computation Over-the-Air: MIMO Beamforming Design](https://mlsanity.com/paper/2201.12581/) (2022)
- [To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks](https://mlsanity.com/paper/2006.08671/) (2020)

## Built on by (most cited first, 15 of 15)

- [PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training](https://mlsanity.com/paper/2407.03361/) (2024)
- [A Survey on Integrated Sensing, Communication, and Computation](https://mlsanity.com/paper/2408.08074/) (2024)
- [CSI-BERT2: A BERT-inspired Framework for Efficient CSI Prediction and Classification in Wireless Communication and Sensing](https://mlsanity.com/paper/2412.06861/) (2024)
- [CrossFi: A Cross Domain Wi-Fi Sensing Framework Based on Siamese Network](https://mlsanity.com/paper/2408.10919/) (2024)
- [A Survey on Wi-Fi Sensing Generalizability: Taxonomy, Techniques, Datasets, and Future Research Prospects](https://mlsanity.com/paper/2503.08008/) (2025)
- [KNN-MMD: Cross Domain Wireless Sensing via Local Distribution Alignment](https://mlsanity.com/paper/2412.04783/) (2024)
- [LoFi: Vision-Aided Label Generator for Wi-Fi Localization and Tracking](https://mlsanity.com/paper/2412.05074/) (2024)
- [Label Unbalance in High-frequency Trading](https://mlsanity.com/paper/2503.09988/) (2025)
- [A Short Overview of Multi-Modal Wi-Fi Sensing](https://mlsanity.com/paper/2505.06682/) (2025)
- [Transforming Decoder-Only Transformers for Accurate WiFi-Telemetry Based Indoor Localization](https://mlsanity.com/paper/2505.15835/) (2025)
- [Cross-Domain Multi-Person Human Activity Recognition via Near-Field Wi-Fi Sensing](https://mlsanity.com/paper/2510.17816/) (2025)
- [Sensing and Understanding the World over Air: A Large Multimodal Model for Mobile Networks](https://mlsanity.com/paper/2511.21707/) (2025)
- [FuseFi: Combining Irregularly Sampled CSI from Diverse Communication Packets and Frequency Bands for Wi-Fi Sensing](https://mlsanity.com/paper/2512.22143/) (2025)
- [Multi-Station WiFi CSI Sensing Framework Robust to Station-wise Feature Missingness and Limited Labeled Data](https://mlsanity.com/paper/2603.11858/) (2026)
- [OmniLoc: A Geometry-Aware Foundation Model for Anchor-Free UE Localization Across Diverse Indoor Environments](https://mlsanity.com/paper/2606.11490/) (2026)

## Abstract (arXiv)

Despite the development of various deep learning methods for Wi-Fi sensing, package loss often results in noncontinuous estimation of the Channel State Information (CSI), which negatively impacts the performance of the learning models. To overcome this challenge, we propose a deep learning model based on Bidirectional Encoder Representations from Transformers (BERT) for CSI recovery, named CSI-BERT. CSI-BERT can be trained in an self-supervised manner on the target dataset without the need for additional data. Furthermore, unlike traditional interpolation methods that focus on one subcarrier at a time, CSI-BERT captures the sequential relationships across different subcarriers. Experimental results demonstrate that CSI-BERT achieves lower error rates and faster speed compared to traditional interpolation methods, even when facing with high loss rates. Moreover, by harnessing the recovered CSI obtained from CSI-BERT, other deep learning models like Residual Network and Recurrent Neural Network can achieve an average increase in accuracy of approximately 15% in Wi-Fi sensing tasks. The collected dataset WiGesture and code for our model are publicly available at https://github.com/RS2002/CSI-BERT.