mlsanityData to 14 Oct 2026

Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction

arXiv 2610.03822 ↗First posted 2 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 41 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

36 of 41 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  7. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  9. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  11. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
24 more
  1. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  2. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
  3. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
  4. 2019DeepJSCC-f: Deep Joint Source-Channel Coding of Images with Feedbackcited by 96
  5. 2021Nonlinear Transform Source-Channel Coding for Semantic Communicationscited by 76
  6. 2022Improved Masked Image Generation with Token-Criticcited by 64
  7. 2023CDDM: Channel Denoising Diffusion Models for Wireless Semantic Communicationscited by 59
  8. 2022Generative Joint Source-Channel Coding for Semantic Image Transmissioncited by 58
  9. 2022Robust Semantic Communications with Masked VQ-VAE Enabled Codebookcited by 56
  10. 2023SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Codingcited by 45
  11. 2020Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Codingcited by 44
  12. 2022DeepJSCC-Q: Constellation Constrained Deep Joint Source-Channel Codingcited by 38
  13. 2022WITT: A Wireless Image Transmission Transformer for Semantic Communicationscited by 37
  14. 2025Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communicationscited by 35
  15. 2023Deep Joint Source-Channel Coding for Adaptive Image Transmission over MIMO Channelscited by 28
  16. 2021OFDM-guided Deep Joint Source Channel Coding for Wireless Multipath Fading Channelscited by 26
  17. 2022Channel-Adaptive Wireless Image Transmission with OFDMcited by 20
  18. 2023Joint Coding-Modulation for Digital Semantic Communications via Variational Autoencodercited by 18
  19. 2025Semantics-Guided Diffusion for Deep Joint Source-Channel Coding in Wireless Image Transmissioncited by 16
  20. 2024DiffCom: Channel Received Signal is a Natural Condition to Guide Diffusion Posterior Samplingcited by 11
  21. 2024From Analog to Digital: Multi-Order Digital Joint Coding-Modulation for Semantic Communicationcited by 9
  22. 2023DeepJSCC-l++: Robust and Bandwidth-Adaptive Wireless Image Transmissioncited by 8
  23. 2024Learned Image Transmission with Hierarchical Variational Autoencodercited by 2
  24. 2026TONIC: Token-Centric Semantic Communication for Task-Oriented Wireless Systemscited by 1

Built on by

Not cited yet.