Lost in Motion: Vision Language Models Fail the Dynamic Gauges Test
On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection
Abstract (arXiv)
The digital transformation of industrial manufacturing increasingly relies on the ability of autonomous robots to interact with legacy infrastructure, particularly analog gauges. Vision-Language Models (VLMs) have the potential to provide a general solution for gauge reading and have already shown good performance in instrument recognition. However, performing accurate, real-time gauge readings is a more complex task. This paper evaluates state-of-the-art models, including versions from the GPT-5 (5.4 Thinking and 5.3 Instant) and Gemini 3 (Pro and Flash) families, against a set of simple but realistic dynamic gauge reading scenarios. To facilitate this evaluation, we introduce a novel dataset comprising video sequences of three instruments of different gauge types: circular, linear, and Vernier, under diverse motion and speed profiles. Our findings indicate that the evaluated frontier VLMs, under our specific testing conditions, exhibit a limited ability to interpret needle trajectories and scale semantics, failing to provide the traceability and reliability needed for safety-critical monitoring. The results demonstrate that these models have not yet achieved the performance necessary to be classified as trustworthy synthetic instruments under existing IEEE and ISO standards.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 5 of its 10 references that are in the map. First posted on arXiv 19 Apr 2026.
Builds on
5 of 10 references.
References · 5
- 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 147
- 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 66
- 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoningcited by 49
- 2025Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBenchcited by 6
- 2023Learning to Read Analog Gauges from Synthetic Datacited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Vision-Language Models for Analog Gauge Reading: An Empirical Study of Specialization, Transfer and Reliability
- 2026InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models
- 2025iSafetyBench: A video-language benchmark for safety in industrial environment
- 2026From Foundation to Application: Improving VLA Models in Practice
- 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
- 2025Transferring Vision-Language-Action Models to Industry Applications: Architectures, Performance, and Challenges
- 2026Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform
- 2026Enabling Dynamic Tracking in Vision-Language-Action Models via Time-Discrete and Time-Continuous Velocity Feedforward