# edgeVLM: Cloud-edge Collaborative Real-time VLM based on Context Transfer

arXiv: https://arxiv.org/abs/2508.12638 · HTML: https://mlsanity.com/paper/2508.12638/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 16 of 16)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025)
- [Visual Planning: Let's Think Only with Images](https://mlsanity.com/paper/2505.11409/) (2025)
- [MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse](https://mlsanity.com/paper/2503.18470/) (2025)
- [A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs](https://mlsanity.com/paper/2412.03324/) (2024)
- [IPN Hand: A Video Dataset and Benchmark for Real-Time Continuous Hand Gesture Recognition](https://mlsanity.com/paper/2005.02134/) (2020)
- [Can LVLMs Obtain a Driver's License? A Benchmark Towards Reliable AGI for Autonomous Driving](https://mlsanity.com/paper/2409.02914/) (2024)
- [PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability](https://mlsanity.com/paper/2503.08481/) (2025)
- [Generative Planning with 3D-vision Language Pre-training for End-to-End Autonomous Driving](https://mlsanity.com/paper/2501.08861/) (2025)
- [Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts](https://mlsanity.com/paper/2501.04322/) (2025)
- [Making Large Vision Language Models to be Good Few-shot Learners](https://mlsanity.com/paper/2408.11297/) (2024)
- [Context-Aware LLM Translation System Using Conversation Summarization and Dialogue History](https://mlsanity.com/paper/2410.16775/) (2024)
- [History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigation](https://mlsanity.com/paper/2506.16623/) (2025)

## Built on by (most cited first, 3 of 3)

- [AVERY: Intent-Driven Adaptive VLM Split Computing via Embodied Self-Awareness for Efficient Disaster Response Systems](https://mlsanity.com/paper/2511.18151/) (2025)
- [Vision Token Manipulation Attacks on Cloud-Edge Inference of Large Vision-Language Models](https://mlsanity.com/paper/2607.02819/) (2026)
- [TALSC: Timeliness-Aware Large-Small VLM Collaboration for Infrastructure-Assisted Autonomous Driving](https://mlsanity.com/paper/2608.01998/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) are increasingly deployed in real-time applications such as autonomous driving and human-computer interaction, which demand fast and reliable responses based on accurate perception. To meet these requirements, existing systems commonly employ cloud-edge collaborative architectures, such as partitioned Large Vision-Language Models (LVLMs) or task offloading strategies between Large and Small Vision-Language Models (SVLMs). However, these methods fail to accommodate cloud latency fluctuations and overlook the full potential of delayed but accurate LVLM responses. In this work, we propose a novel cloud-edge collaborative paradigm for VLMs, termed Context Transfer, which treats the delayed outputs of LVLMs as historical context to provide real-time guidance for SVLMs inference. Based on this paradigm, we design edgeVLM, which incorporates both context replacement and visual focus modules to refine historical textual input and enhance visual grounding consistency. Extensive experiments on three real-time vision-lanuage reasoning tasks across four datasets demonstrate the effectiveness of the proposed framework. The new paradigm lays the groundwork for more effective and latency-aware collaboration strategies in future VLM systems.