# Vision Transformer-based Semantic Communications With Importance-Aware Quantization

arXiv: https://arxiv.org/abs/2412.06038 · HTML: https://mlsanity.com/paper/2412.06038/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Builds on (most cited first, 25 of 25)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Domain Generalization by Mutual-Information Regularization with Pre-trained Models](https://mlsanity.com/paper/2203.10789/) (2022)
- [Semantic Communication Systems for Speech Transmission](https://mlsanity.com/paper/2102.12605/) (2021)
- [Generative Joint Source-Channel Coding for Semantic Image Transmission](https://mlsanity.com/paper/2211.13772/) (2022)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Task-Oriented Communication for Multi-Device Cooperative Edge Inference](https://mlsanity.com/paper/2109.00172/) (2021)
- [Task-Oriented Multi-User Semantic Communications for VQA Task](https://mlsanity.com/paper/2108.07357/) (2021)
- [Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2009.12480/) (2020)
- [Resource allocation for text semantic communications](https://mlsanity.com/paper/2201.06023/) (2022)
- [D2-JSCC: Digital Deep Joint Source-channel Coding for Semantic Communications](https://mlsanity.com/paper/2403.07338/) (2024)
- [Joint Source-Channel Coding for Channel-Adaptive Digital Semantic Communications](https://mlsanity.com/paper/2311.08146/) (2023)
- [Digital-SC: Digital Semantic Communication with Adaptive Network Split and Learned Non-Linear Quantization](https://mlsanity.com/paper/2305.13553/) (2023)
- [Goal-Oriented Communications for the IoT and Application to Data Compression](https://mlsanity.com/paper/2211.05378/) (2022)
- [OFDM-Based Digital Semantic Communication with Importance Awareness](https://mlsanity.com/paper/2401.02178/) (2024)
- [Vision Transformer for Adaptive Image Transmission over MIMO Channels](https://mlsanity.com/paper/2210.15347/) (2022)
- [The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers](https://mlsanity.com/paper/2302.10494/) (2023)
- [A Unified Multi-Task Semantic Communication System with Domain Adaptation](https://mlsanity.com/paper/2206.00254/) (2022)
- [Attention-aware Semantic Communications for Collaborative Inference](https://mlsanity.com/paper/2404.07217/) (2024)
- [Reconfigurable Intelligent Surfaces Relying on Non-Diagonal Phase Shift Matrices](https://mlsanity.com/paper/2203.08184/) (2022)
- [Deep Joint Source-Channel Coding for Wireless Image Transmission with Semantic Importance](https://mlsanity.com/paper/2302.02287/) (2023)

## Built on by (most cited first, 6 of 6)

- [Importance-Aware Semantic Communication in MIMO-OFDM Systems Using Vision Transformer](https://mlsanity.com/paper/2508.07696/) (2025)
- [Collaborative Edge-to-Server Inference for Vision-Language Models](https://mlsanity.com/paper/2512.16349/) (2025)
- [Quantization-Aware Collaborative Inference for Large Embodied AI Models](https://mlsanity.com/paper/2602.13052/) (2026)
- [Context-Aware Wireless Token Communication via Joint Token Masking and Detection](https://mlsanity.com/paper/2605.02123/) (2026)
- [SAGE: Training-Free Semantic Evidence Composition for Edge-Cloud Inference under Hard Uplink Budgets](https://mlsanity.com/paper/2604.19623/) (2026)
- [Token-Oriented Semantic Communication with Pretrained Vision Transformers](https://mlsanity.com/paper/2608.25410/) (2026)

## Abstract (arXiv)

Semantic communications provide significant performance gains over traditional communications by transmitting task-relevant semantic features through wireless channels. However, most existing studies rely on end-to-end (E2E) training of neural-type encoders and decoders to ensure effective transmission of these semantic features. To enable semantic communications without relying on E2E training, this paper presents a vision transformer (ViT)-based semantic communication system with importance-aware quantization (IAQ) for wireless image transmission. The core idea of the presented system is to leverage the attention scores of a pretrained ViT model to quantify the importance levels of image patches. Based on this idea, our IAQ framework assigns different quantization bits to image patches based on their importance levels. This is achieved by formulating a weighted quantization error minimization problem, where the weight is set to be an increasing function of the attention score. Then, an optimal incremental allocation method and a low-complexity water-filling method are devised to solve the formulated problem. Our framework is further extended for realistic digital communication systems by modifying the bit allocation problem and the corresponding allocation methods based on an equivalent binary symmetric channel (BSC) model. Simulations on single-view and multi-view image classification tasks show that our IAQ framework outperforms conventional image compression methods in both error-free and realistic communication scenarios.