mlsanityData to 7 Oct 2026

Vision Token Manipulation Attacks on Cloud-Edge Inference of Large Vision-Language Models

arXiv 2607.02819 ↗First posted 2 Jul 2026cs.CR cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Cloud-edge Large Vision-Language Model (LVLM) inference enables efficient deployment by splitting computation between edge devices and cloud servers. In this process, intermediate vision tokens are transmitted from the edge to the cloud over a communication link, thereby exposing a new attack surface. We study vision token manipulation attack (VTM-Attack) under a black-box man-in-the-middle setting, where an adversary intercepts and manipulates a subset of transmitted vision tokens under a budget constraint. We propose four na"ive attack strategies and an optimization-based token selection method. Experiments on 6 state-of-the-art LVLMs (3B-72B) across 4 benchmarks show that manipulating only 10% of vision tokens can reduce accuracy by up to 88.31%. These results reveal a critical vulnerability in cloud-edge LVLM inference.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 17 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

14 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025BackdoorVLM: A Benchmark for Backdoor Attacks and Defenses on Vision-Language Models24 Nov

Other references · 13

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  4. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  6. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  7. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  8. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  9. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  10. 2025Prompt Inference Attack on Distributed Large Language Model Inference Frameworkscited by 13
  11. 2023On the Robustness of Split Learning against Adversarial Attackscited by 6
  12. 2025edgeVLM: Cloud-edge Collaborative Real-time VLM based on Context Transfercited by 3
1 more
  1. 2025Hyperion: Low-Latency Ultra-HD Video Analytics via Collaborative Vision Transformer Inferencecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
  2. 2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models
  3. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulated
  4. 2026Less Is More -- Until It Breaks: Security Pitfalls of Vision Token Compression in Large Vision-Language Models
  5. 2025When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs
  6. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images
  7. 2026On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline
  8. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations