Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms
New in Multimodal LLMs / vision-language models; no direction yet. Multimodal LLMs / vision-language models
Abstract (arXiv)
Vision-language models (VLMs) are moving onto edge devices, but little is known about how much energy they draw on single-board computers with NPU or GPU accelerators, or where that energy goes. We measure total-system energy, latency, and thermal behavior of Qwen2-VL-2B-Instruct and its text-only sibling on a Raspberry Pi 5 with a Hailo-10H NPU and on a Jetson Orin Nano Super, on both CPU and accelerator, and separate the fixed energy of each request from the energy that grows with output length. Because the accelerators use different quantization and serving software, we compare deployment stacks rather than silicon. Three findings stand out. First, NPU offloading makes the Pi a much more efficient VLM platform: it cuts energy per generated token by about two-thirds, leaves the host CPU largely free, and avoids the thermal throttling that the CPU backend reaches in sustained use. Second, an image adds both fixed and per-token energy, but at typical response lengths generating the answer costs far more than processing the image. Third, the Jetson GPU favors short responses through its lower fixed cost per request, whereas the Pi NPU favors longer ones through its lower cost per output token; their energy curves cross near 40 output tokens.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 14 Oct 2026.
Builds on
16 of 26 references.
References · 16
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024Qwen2 Technical Reportcited by 2,120
- 2019MLPerf Inference Benchmarkcited by 199
- 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 170
- 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 128
- 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devicescited by 122
- 2024Small Language Models: Survey, Measurements, and Insightscited by 112
- 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 55
- 2024MELTing point: Mobile Evaluation of Language Transformerscited by 36
- 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 34
- 2024The Price of Prompting: Profiling Energy Use in Large Language Models Inferencecited by 23
- 2024MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Casescited by 14
4 more
- 2024PalmBench: A Comprehensive Benchmark of Compressed Large Language Models on Mobile Platformscited by 8
- 2026LLM Inference at the Edge: Mobile, NPU, and GPU Performance Efficiency Trade-offs Under Sustained Loadcited by 4
- 2023Accurate Calibration of Power Measurements from Internal Power Sensors on NVIDIA Jetson Devicescited by 2
- 2026Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inferencecited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Scaling Laws for Energy Efficiency of Local LLMs
- 2026Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
- 2025Cross-Platform Scaling of Vision-Language-Action Models from Edge to Cloud GPUs
- 2025Understanding the Performance and Power of LLM Inferencing on Edge Accelerators
- 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architectures
- 2026Understanding Efficiency: Quantization, Batching, and Serving Strategies in LLM Energy Use
- 2026A Picture is Worth a Thousand Tokens: How Vision Language Models Cut AI Energy Costs While Improving Accuracy
- 2026Energy Efficiency of Locally Deployed LLMs: A Preliminary Quantitative GPU Power Benchmark on Consumer Hardware