# Context-Aware Wireless Token Communication via Joint Token Masking and Detection

arXiv: https://arxiv.org/abs/2605.02123 · HTML: https://mlsanity.com/paper/2605.02123/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Large Language Model Enhanced Multi-Agent Systems for 6G Communications](https://mlsanity.com/paper/2312.07850/) (2023)
- [Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications](https://mlsanity.com/paper/2502.12096/) (2025)
- [Adaptive Semantic Token Communication for Transformer-based Edge Inference](https://mlsanity.com/paper/2505.17604/) (2025)
- [Large-Language-Model Enabled Semantic Communication Systems](https://mlsanity.com/paper/2407.14112/) (2024)
- [Leveraging Large Language Models for Wireless Symbol Detection via In-Context Learning](https://mlsanity.com/paper/2409.00124/) (2024)
- [Attention-aware Semantic Communications for Collaborative Inference](https://mlsanity.com/paper/2404.07217/) (2024)
- [Vision Transformer-based Semantic Communications With Importance-Aware Quantization](https://mlsanity.com/paper/2412.06038/) (2024)
- [Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission](https://mlsanity.com/paper/2505.11788/) (2025)
- [Low-Complexity Semantic Packet Aggregation for Token Communication via Lookahead Search](https://mlsanity.com/paper/2506.19451/) (2025)
- [Blind Training for Channel-Adaptive Digital Semantic Communications](https://mlsanity.com/paper/2501.02273/) (2025)
- [ESC-MVQ: End-to-End Semantic Communication With Multi-Codebook Vector Quantization](https://mlsanity.com/paper/2504.11709/) (2025)
- [MIMO Detection under Hardware Impairments: Data Augmentation With Boosting](https://mlsanity.com/paper/2412.06049/) (2024)
- [Context-Aware Iterative Token Detection and Masked Transmission for Wireless Token Communication](https://mlsanity.com/paper/2601.17770/) (2026)

## Built on by (most cited first, 1 of 1)

- [From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity](https://mlsanity.com/paper/2609.10714/) (2026)

## Abstract (arXiv)

The increasing use of token-based representations in language-driven applications has motivated wireless token communication, where tokens are treated as fundamental units for transmission. However, conventional communication systems overlook dependencies among tokens and allocate transmission resources uniformly, leading to inefficient use of limited wireless resources under channel impairments. In this paper, we propose a context-aware token communication framework that leverages a masked language model (MLM) as a shared contextual model between the transmitter (Tx) and receiver (Rx). At the Rx, we develop a context-aware token detection method that integrates channel likelihoods with MLM-based contextual priors under a Bayesian formulation, enabling robust token inference over noisy channels. At the Tx, we propose a context-aware token masking strategy that selectively omits tokens that can be reliably inferred at the Rx, allowing the available power budget to be concentrated on more informative tokens. These components are jointly designed through a shared MLM, establishing a unified Tx-Rx framework for efficient token transmission and detection. Simulation results demonstrate that the proposed framework significantly improves reconstruction performance compared to conventional and existing token communication schemes, achieving up to 1.77X and 1.63X performance gains on the Europarl corpus and WikiText-103 datasets, respectively.