State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Reading
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Abstract (arXiv)
Multimodal large language models (MLLMs) have achieved impressive progress on general multimodal tasks, yet they remain brittle on dial-based measurement reading. In this paper, we study this problem through controlled benchmarks and feature-space probing, and show that current MLLMs not only achieve unsatisfactory accuracy on dial-based readout, but also suffer sharp performance drops under viewpoint and illumination changes even when the underlying dial state remains fixed. Our probing analysis further reveals that same-state samples under appearance variation are not consistently clustered, while neighboring states fail to preserve the local structure implied by continuous dial values. These findings suggest that existing MLLMs largely ignore the intrinsic state geometry of dial measurement tasks and instead rely on superficial appearance cues. Motivated by this diagnosis, we propose TriSCA, a tri-level state-consistent alignment framework for dial-based measurement reading. Specifically, TriSCA consists of state-distance-aware representation alignment, metadata-grounded observation-to-state supervision, and state-aware objective alignment. Extensive ablation studies and evaluation experiments on controlled clock and gauge benchmarks, together with evaluation on an external real-world benchmark, demonstrate the effectiveness of our method.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 37 references that are in the map. First posted on arXiv 29 Apr 2026.
Builds on
22 of 37 references.
Closest ancestors · 7
Same direction, newest first
- 2025Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs28 May
- 2025Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?16 May
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceive18 Apr
- 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Models29 Jan
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic27 Jun
Other references · 15
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2019Towards VQA Models That Can Readcited by 1,718
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
- 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
- 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
- 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 68
- 2024VisOnlyQA: Large Vision Language Models Still Struggle with Visual Perception of Geometric Informationcited by 39
- 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 37
- 2021It's About Time: Analog Clock Reading in the Wildcited by 14
- 2026Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoningcited by 11
3 more
- 2020Deep Learning for Image-based Automatic Dial Meter Reading: Dataset and Baselinescited by 9
- 2022Image-based Automatic Dial Meter Reading in Unconstrained Scenarioscited by 2
- 2025Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficultycited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench
- 2025Aligning Multimodal LLM with Human Preference: A Survey
- 2026The Alignment Illusion in Multimodal Large Language Models
- 2024The Revolution of Multimodal Large Language Models: A Survey
- 2025Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs
- 2026Seeing, Saying, but Not Using: From Reportable Spatial Facts to Usable States in Multimodal Large Language Models
- 2024Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models
- 2026COHERENCE: Benchmarking Fine-Grained Image-Text Alignment in Interleaved Multimodal Contexts