# Attention-aware Semantic Communications for Collaborative Inference

arXiv: https://arxiv.org/abs/2404.07217 · HTML: https://mlsanity.com/paper/2404.07217/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [EfficientFormer: Vision Transformers at MobileNet Speed](https://mlsanity.com/paper/2206.01191/) (2022)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019)
- [EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers](https://mlsanity.com/paper/2205.03436/) (2022)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [Towards an Intelligent Edge: Wireless Communication Meets Machine Learning](https://mlsanity.com/paper/1809.00343/) (2018)
- [A Lite Distributed Semantic Communication System for Internet of Things](https://mlsanity.com/paper/2007.11095/) (2020)
- [Communication-Computation Trade-Off in Resource-Constrained Edge Inference](https://mlsanity.com/paper/2006.02166/) (2020)
- [Wireless Image Retrieval at the Edge](https://mlsanity.com/paper/2007.10915/) (2020)
- [From Semantic Communication to Semantic-aware Networking: Model, Architecture, and Open Problems](https://mlsanity.com/paper/2012.15405/) (2020)
- [What is Semantic Communication? A View on Conveying Meaning in the Era of Machine Intelligence](https://mlsanity.com/paper/2110.00196/) (2021)
- [Optimizing Relevance Maps of Vision Transformers Improves Robustness](https://mlsanity.com/paper/2206.01161/) (2022)
- [Making Vision Transformers Efficient from A Token Sparsification View](https://mlsanity.com/paper/2303.08685/) (2023)
- [Data-Importance Aware User Scheduling for Communication-Efficient Edge Machine Learning](https://mlsanity.com/paper/1910.02214/) (2019)
- [Collaborative Inference for AI-Empowered IoT Devices](https://mlsanity.com/paper/2207.11664/) (2022)
- [DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices](https://mlsanity.com/paper/2309.05015/) (2023)
- [Goal-Oriented Communications for the IoT and Application to Data Compression](https://mlsanity.com/paper/2211.05378/) (2022)
- [Wireless Data Acquisition for Edge Learning: Data-Importance Aware Retransmission](https://mlsanity.com/paper/1812.02030/) (2018)
- [Dynamic Compression Ratio Selection for Edge Inference Systems with Hard Deadlines](https://mlsanity.com/paper/2005.12235/) (2020)

## Built on by (most cited first, 7 of 7)

- [Vision Transformer-based Semantic Communications With Importance-Aware Quantization](https://mlsanity.com/paper/2412.06038/) (2024)
- [Importance-Aware Semantic Communication in MIMO-OFDM Systems Using Vision Transformer](https://mlsanity.com/paper/2508.07696/) (2025)
- [Collaborative Edge-to-Server Inference for Vision-Language Models](https://mlsanity.com/paper/2512.16349/) (2025)
- [Context-Aware Wireless Token Communication via Joint Token Masking and Detection](https://mlsanity.com/paper/2605.02123/) (2026)
- [SAGE: Training-Free Semantic Evidence Composition for Edge-Cloud Inference under Hard Uplink Budgets](https://mlsanity.com/paper/2604.19623/) (2026)
- [ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover](https://mlsanity.com/paper/2608.10545/) (2026)
- [Token-Oriented Semantic Communication with Pretrained Vision Transformers](https://mlsanity.com/paper/2608.25410/) (2026)

## Abstract (arXiv)

We propose a communication-efficient collaborative inference framework in the domain of edge inference, focusing on the efficient use of vision transformer (ViT) models. The partitioning strategy of conventional collaborative inference fails to reduce communication cost because of the inherent architecture of ViTs maintaining consistent layer dimensions across the entire transformer encoder. Therefore, instead of employing the partitioning strategy, our framework utilizes a lightweight ViT model on the edge device, with the server deploying a complicated ViT model. To enhance communication efficiency and achieve the classification accuracy of the server model, we propose two strategies: 1) attention-aware patch selection and 2) entropy-aware image transmission. Attention-aware patch selection leverages the attention scores generated by the edge device's transformer encoder to identify and select the image patches critical for classification. This strategy enables the edge device to transmit only the essential patches to the server, significantly improving communication efficiency. Entropy-aware image transmission uses min-entropy as a metric to accurately determine whether to depend on the lightweight model on the edge device or to request the inference from the server model. In our framework, the lightweight ViT model on the edge device acts as a semantic encoder, efficiently identifying and selecting the crucial image information required for the classification task. Our experiments demonstrate that the proposed collaborative inference framework can reduce communication overhead by 68% with only a minimal loss in accuracy compared to the server model on the ImageNet dataset.