# Token Encoding for Semantic Recovery

arXiv: https://arxiv.org/abs/2604.12931 · HTML: https://mlsanity.com/paper/2604.12931/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation](https://mlsanity.com/paper/2403.04692/) (2024)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases](https://mlsanity.com/paper/2402.14905/) (2024)
- [Fast On-device LLM Inference with NPUs](https://mlsanity.com/paper/2407.05858/) (2024)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Generative AI-driven Semantic Communication Networks: Architecture, Technologies and Applications](https://mlsanity.com/paper/2401.00124/) (2023)
- [6G Wireless Systems: Vision, Requirements, Challenges, Insights, and Opportunities](https://mlsanity.com/paper/2008.03213/) (2020)
- [Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications](https://mlsanity.com/paper/2502.12096/) (2025)
- [Internet of Underwater Things and Big Marine Data Analytics -- A Comprehensive Survey](https://mlsanity.com/paper/2012.06712/) (2020)
- [GRACE: Loss-Resilient Real-Time Video through Neural Codecs](https://mlsanity.com/paper/2305.12333/) (2023)
- [Generative Semantic Communication via Textual Prompts: Latency Performance Tradeoffs](https://mlsanity.com/paper/2409.09715/) (2024)
- [Semantic Satellite Communications Based on Generative Foundation Model](https://mlsanity.com/paper/2404.11941/) (2024)
- [Towards the Internet of Underground Things: A Systematic Survey](https://mlsanity.com/paper/1902.03844/) (2019)
- [ResiTok: A Resilient Tokenization-Enabled Framework for Ultra-Low-Rate and Robust Image Transmission](https://mlsanity.com/paper/2505.01870/) (2025)
- [Distillation-Enabled Knowledge Alignment Protocol for Semantic Communication in AI Agent Networks](https://mlsanity.com/paper/2505.17030/) (2025)
- [Distillation-Enabled Knowledge Alignment for Generative Semantic Communications of AIGC Images](https://mlsanity.com/paper/2506.19893/) (2025)
- [Error-Resilient Semantic Communication for Speech Transmission over Packet-Loss Networks](https://mlsanity.com/paper/2512.08203/) (2025)

## Built on by (most cited first, 1 of 1)

- [From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity](https://mlsanity.com/paper/2609.10714/) (2026)

## Abstract (arXiv)

In generative semantic communication, semantic tokens guide receiver-side generative models to synthesize high-dimensional content. In challenging network environments, however, frequent token erasure distorts the conveyed semantics beyond what receiver-side recovery can restore. In this paper, we propose a token encoding framework (TokCode) for robust semantic recovery, achieving erasure resilience by restructuring redundancy in the semantic domain. TokCode uses a lightweight adapter to recast a general-purpose large language model (LLM) at the transmitter into a token encoder, exploiting the LLM's pretrained semantic prior to avoid introducing a dedicated deep model. To optimize the adapter efficiently and make it applicable across diverse channels, we develop a channel-quality-aware distillation approach for token encoder training~(CADET). Using a differentiable sentence-level semantic surrogate, CADET tunes T5 foundation models into experts for distinct erasure rates and distills them into a single reconfigurable low-rank adapter, enabling subsequent reinforcement learning (RL) to start above the plateau where direct RL stalls. Simulation results on token-based generative image transmission show that TokCode improves the image-level similarity over the best-performing receiver-side recovery benchmark by 14.1%--22.4%, closing 71.9%--76.5% of its gap to the erasure-aware oracle encoding, when only 20% to 50% of the tokens survive.