# Semantic Edge-Cloud Communication for Real-Time Urban Traffic Surveillance with ViT and LLMs over Mobile Networks arXiv: https://arxiv.org/abs/2509.21259 ยท HTML: https://mlsanity.com/paper/2509.21259/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 47 of its 74 references that are in the map. - First posted on arXiv 25 Sep 2025. ## Placed in - Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/) ## Builds on (most cited first, 30 of 47) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024) - [DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving](https://mlsanity.com/paper/2312.09245/) (2023) - [Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2311.13165/) (2023) - [Semantic Communications: Principles and Challenges](https://mlsanity.com/paper/2201.01389/) (2021) - [Large AI Model Empowered Multimodal Semantic Communications](https://mlsanity.com/paper/2309.01249/) (2023) - [WITT: A Wireless Image Transmission Transformer for Semantic Communications](https://mlsanity.com/paper/2211.00937/) (2022) - [Towards Semantic Communications: Deep Learning-Based Image Semantic Coding](https://mlsanity.com/paper/2208.04094/) (2022) - [Personalized Saliency in Task-Oriented Semantic Communications: Image Transmission and Performance Analysis](https://mlsanity.com/paper/2209.12274/) (2022) - [Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles](https://mlsanity.com/paper/2205.14942/) (2022) - [Semantic Communications for Image Recovery and Classification via Deep Joint Source and Channel Coding](https://mlsanity.com/paper/2304.02317/) (2023) - [UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding](https://mlsanity.com/paper/2506.23219/) (2025) - [Wireless End-to-End Image Transmission System using Semantic Communications](https://mlsanity.com/paper/2302.13721/) (2023) - [DeepMA: End-to-end Deep Multiple Access for Wireless Image Transmission in Semantic Communication](https://mlsanity.com/paper/2303.11543/) (2023) - [Generative Model Based Highly Efficient Semantic Communication Approach for Image Transmission](https://mlsanity.com/paper/2211.10287/) (2022) - [YOLO-based Semantic Communication with Generative AI-aided Resource Allocation for Digital Twins Construction](https://mlsanity.com/paper/2306.14138/) (2023) - [Demo: Real-Time Semantic Communications with a Vision Transformer](https://mlsanity.com/paper/2205.03886/) (2022) - [Generative AI-driven Semantic Communication Framework for NextG Wireless Network](https://mlsanity.com/paper/2310.09021/) (2023) - [A Multi-Task Oriented Semantic Communication Framework for Autonomous Vehicles](https://mlsanity.com/paper/2403.12997/) (2024) - [Optimization of Image Transmission in a Cooperative Semantic Communication Networks](https://mlsanity.com/paper/2301.00433/) (2023) - [On the Role of ViT and CNN in Semantic Communications: Analysis and Prototype Validation](https://mlsanity.com/paper/2306.02759/) (2023) - [Scenario Understanding of Traffic Scenes Through Large Visual Language Models](https://mlsanity.com/paper/2501.17131/) (2025) - [Edge Learning for 6G-enabled Internet of Things: A Comprehensive Survey of Vulnerabilities, Datasets, and Defenses](https://mlsanity.com/paper/2306.10309/) (2023) - [Importance-Aware Image Segmentation-based Semantic Communication for Autonomous Driving](https://mlsanity.com/paper/2401.10153/) (2024) - [Large Language Models (LLMs) for Semantic Communication in Edge-based IoT Networks](https://mlsanity.com/paper/2407.20970/) (2024) - [Generative AI-Enhanced Multi-Modal Semantic Communication in Internet of Vehicles: System Design and Methodologies](https://mlsanity.com/paper/2409.15642/) (2024) - [How to Evaluate Semantic Communications for Images with ViTScore Metric?](https://mlsanity.com/paper/2309.04891/) (2023) ## Built on by (most cited first, 3 of 3) - [Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction](https://mlsanity.com/paper/2601.17216/) (2026) - [A Lightweight Digital-Twin-Based Framework for Edge-Assisted Vehicle Tracking and Collision Prediction](https://mlsanity.com/paper/2603.07338/) (2026) - [Semantic Edge-Cloud Communication for Real-Time Urban Traffic Digital Twins](https://mlsanity.com/paper/2610.14187/) (2026) ## Abstract (arXiv) Real-time urban traffic surveillance is vital for Intelligent Transportation Systems (ITS) to ensure road safety, optimize traffic flow, track vehicle trajectories, and prevent collisions in smart cities. Deploying edge cameras across urban environments is a standard practice for monitoring road conditions. However, integrating these with intelligent models requires a robust understanding of dynamic traffic scenarios and a responsive interface for user interaction. Although multimodal Large Language Models (LLMs) can interpret traffic images and generate informative responses, their deployment on edge devices is infeasible due to high computational demands. Therefore, LLM inference must occur on the cloud, necessitating visual data transmission from edge to cloud, a process hindered by limited bandwidth, leading to potential delays that compromise real-time performance. To address this challenge, we propose a semantic communication framework that significantly reduces transmission overhead. Our method involves detecting Regions of Interest (RoIs) using YOLOv11, cropping relevant image segments, and converting them into compact embedding vectors using a Vision Transformer (ViT). These embeddings are then transmitted to the cloud, where an image decoder reconstructs the cropped images. The reconstructed images are processed by a multimodal LLM to generate traffic condition descriptions. This approach achieves a 99.9% reduction in data transmission size while maintaining an LLM response accuracy of 89% for reconstructed cropped images, compared to 93% accuracy with original cropped images. Our results demonstrate the efficiency and practicality of ViT and LLM-assisted edge-cloud semantic communication for real-time traffic surveillance.