mlsanityData to 14 Oct 2026

Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?

arXiv 2505.10862 ↗First posted 16 May 2025cs.CLcited by 4

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models which can answer complex questions on an image struggle to tell the time on analog clocks. This is probably due to the lack of images with clocks at different times in their training set. In this work we explore this issue with one of the latest MLLMs: GPT-4.1 to understand why MLLMs fail to tell the time and whether fine-tuning can solve the problem. The results show how models are making progress in reading the time on analog clocks. But have they really learned to do it, or have they only learned patterns in their training datasets? In this work we put the models to the test with different clocks to illustrate the limitations of MLLMs to abstract and generalize.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 5 of its 8 references that are in the map. First posted on arXiv 16 May 2025.

Builds on

5 of 8 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation8 Apr
  2. 2023A Survey on Multimodal Large Language Models23 Jun

Other references · 3

  1. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  2. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  3. 2025Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Disentangling Perception and Reasoning in Multimodal LLMs via Reward Designcited by 3
  2. 2026It's Time to Get It Right: Improving Analog Clock Reading and Clock-Hand Spatial Reasoning in Vision-Language Modelscited by 1
  3. 2026State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Readingcited by 0
  4. 2026ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Can Multimodal LLMs do Visual Temporal Understanding and Reasoning? The answer is No!
  2. 2025Evaluating Graphical Perception with Multimodal LLMs
  3. 2025MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis
  4. 2024Hallucination of Multimodal Large Language Models: A Survey
  5. 2024A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks
  6. 2025Benchmarking Large and Small MLLMs
  7. 2024A Survey on Benchmarks of Multimodal Large Language Models
  8. 2024Chrono: A Simple Blueprint for Representing Time in MLLMs