Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction
Not on the map. Why?
Abstract (arXiv)
Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 41 references that are in the map. First posted on arXiv 2 Oct 2026.
Builds on
36 of 41 references.
References · 36
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2017Neural Discrete Representation Learningcited by 4,712
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
- 2022MaskGIT: Masked Generative Image Transformercited by 1,099
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
24 more
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
- 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
- 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
- 2019DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedbackcited by 96
- 2021Nonlinear Transform Source-Channel Coding for Semantic Communicationscited by 76
- 2022Improved Masked Image Generation with Token-Criticcited by 64
- 2023CDDM: Channel Denoising Diffusion Models for Wireless Semantic Communicationscited by 59
- 2022Generative Joint Source-Channel Coding for Semantic Image Transmissioncited by 58
- 2022Robust Semantic Communications with Masked VQ-VAE Enabled Codebookcited by 56
- 2023SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Codingcited by 45
- 2020Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Codingcited by 44
- 2022DeepJSCC-Q: Constellation Constrained Deep Joint Source-Channel Codingcited by 38
- 2022WITT: A Wireless Image Transmission Transformer for Semantic Communicationscited by 37
- 2025Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communicationscited by 35
- 2023Deep Joint Source-Channel Coding for Adaptive Image Transmission over MIMO Channelscited by 28
- 2021OFDM-guided Deep Joint Source Channel Coding for Wireless Multipath Fading Channelscited by 26
- 2022Channel-Adaptive Wireless Image Transmission with OFDMcited by 20
- 2023Joint Coding-Modulation for Digital Semantic Communications via Variational Autoencodercited by 18
- 2025Semantics-Guided Diffusion for Deep Joint Source-Channel Coding in Wireless Image Transmissioncited by 16
- 2024DiffCom: Channel Received Signal is a Natural Condition to Guide Diffusion Posterior Samplingcited by 11
- 2024From Analog to Digital: Multi-Order Digital Joint Coding-Modulation for Semantic Communicationcited by 9
- 2023DeepJSCC-l++: Robust and Bandwidth-Adaptive Wireless Image Transmissioncited by 8
- 2024Learned Image Transmission with Hierarchical Variational Autoencodercited by 2
- 2026TONIC: Token-Centric Semantic Communication for Task-Oriented Wireless Systemscited by 1
Built on by
Not cited yet.