# Adaptive Semantic Token Communication for Transformer-based Edge Inference

arXiv: https://arxiv.org/abs/2505.17604 · HTML: https://mlsanity.com/paper/2505.17604/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 57 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 30 of 43)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [AdaViT: Adaptive Vision Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2111.15668/) (2021)
- [Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation](https://mlsanity.com/paper/2004.09602/) (2020)
- [MoEfication: Transformer Feed-forward Layers are Mixtures of Experts](https://mlsanity.com/paper/2110.01786/) (2021)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Knowledge Distillation from Internal Representations](https://mlsanity.com/paper/1910.03723/) (2019)
- [Nonlinear Transform Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2112.10961/) (2021)
- [SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2308.09361/) (2023)
- [Deep Joint Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2211.08747/) (2022)
- [Goal-Oriented and Semantic Communication in 6G AI-Native Networks: The 6G-GOALS Approach](https://mlsanity.com/paper/2402.07573/) (2024)
- [Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications](https://mlsanity.com/paper/2502.12096/) (2025)
- [Joint Source-Channel Coding: Fundamentals and Recent Progress in Practical Designs](https://mlsanity.com/paper/2409.17557/) (2024)
- [Deep Joint Source-Channel Coding for Wireless Image Transmission with Adaptive Rate Control](https://mlsanity.com/paper/2110.04456/) (2021)
- [Semantic Communication with Adaptive Universal Transformer](https://mlsanity.com/paper/2108.09119/) (2021)
- [Task-Oriented Communications for NextG: End-to-End Deep Learning and AI Security Aspects](https://mlsanity.com/paper/2212.09668/) (2022)
- [Goal-oriented Communications for the IoT: System Design and Adaptive Resource Optimization](https://mlsanity.com/paper/2310.13948/) (2023)
- [Token-Domain Multiple Access: Exploiting Semantic Orthogonality for Collision Mitigation](https://mlsanity.com/paper/2502.06118/) (2025)

## Built on by (most cited first, 14 of 14)

- [Low-Complexity Semantic Packet Aggregation for Token Communication via Lookahead Search](https://mlsanity.com/paper/2506.19451/) (2025)
- [Video TokenCom: Textual Intent-Guided Multi-Rate Video Token Communications with UEP-Based Adaptive Source-Channel Coding](https://mlsanity.com/paper/2603.02470/) (2026)
- [RIS-aided Latent Space Alignment for Semantic Channel Equalization](https://mlsanity.com/paper/2507.16450/) (2025)
- [Communication Efficient Split Learning of ViTs with Attention-based Double Compression](https://mlsanity.com/paper/2509.15058/) (2025)
- [Context-Aware Wireless Token Communication via Joint Token Masking and Detection](https://mlsanity.com/paper/2605.02123/) (2026)
- [Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication](https://mlsanity.com/paper/2608.16192/) (2026)
- [Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation](https://mlsanity.com/paper/2608.28086/) (2026)
- [EPARA: Parallelizing Categorized AI Inference in Edge Clouds](https://mlsanity.com/paper/2511.00603/) (2025)
- [Evolving Token Communication with Parametric Memory Network](https://mlsanity.com/paper/2605.01869/) (2026)
- [Explainable Task-Oriented Token Communication for AI-Native 6G Networks](https://mlsanity.com/paper/2606.14808/) (2026)
- [Exploiting Overlapping Fields of View for Redundancy-Aware Uplink Transmission in Vehicular 6G](https://mlsanity.com/paper/2606.31715/) (2026)
- [Token-Oriented Semantic Communication with Pretrained Vision Transformers](https://mlsanity.com/paper/2608.25410/) (2026)
- [From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivity](https://mlsanity.com/paper/2609.10714/) (2026)
- [Queue-Aware Token Communications: Bridging AI Representation and Wireless Service](https://mlsanity.com/paper/2610.16730/) (2026)

## Abstract (arXiv)

This paper presents an adaptive framework for edge inference based on a dynamically configurable transformer-powered deep joint source channel coding (DJSCC) architecture. Motivated by a practical scenario where a resource constrained edge device engages in goal oriented semantic communication, such as selectively transmitting essential features for object detection to an edge server, our approach enables efficient task aware data transmission under varying bandwidth and channel conditions. To achieve this, input data is tokenized into compact high level semantic representations, refined by a transformer, and transmitted over noisy wireless channels. As part of the DJSCC pipeline, we employ a semantic token selection mechanism that adaptively compresses informative features into a user specified number of tokens per sample. These tokens are then further compressed through the JSCC module, enabling a flexible token communication strategy that adjusts both the number of transmitted tokens and their embedding dimensions. We incorporate a resource allocation algorithm based on Lyapunov stochastic optimization to enhance robustness under dynamic network conditions, effectively balancing compression efficiency and task performance. Experimental results demonstrate that our system consistently outperforms existing baselines, highlighting its potential as a strong foundation for AI native semantic communication in edge intelligence applications.