# Have Multimodal Large Language Models (MLLMs) Really Learned to Tell the Time on Analog Clocks?

arXiv: https://arxiv.org/abs/2505.10862 · HTML: https://mlsanity.com/paper/2505.10862/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 5 of its 8 references that are in the map.
- First posted on arXiv 16 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 5 of 5)

- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Scaling Relationship on Learning Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2308.01825/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation](https://mlsanity.com/paper/2404.05674/) (2024)
- [Lost in Time: Clock and Calendar Understanding Challenges in Multimodal LLMs](https://mlsanity.com/paper/2502.05092/) (2025)

## Built on by (most cited first, 4 of 4)

- [Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design](https://mlsanity.com/paper/2601.00215/) (2026)
- [It's Time to Get It Right: Improving Analog Clock Reading and Clock-Hand Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2603.08011/) (2026)
- [State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Reading](https://mlsanity.com/paper/2604.26614/) (2026)
- [ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training](https://mlsanity.com/paper/2609.16639/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models which can answer complex questions on an image struggle to tell the time on analog clocks. This is probably due to the lack of images with clocks at different times in their training set. In this work we explore this issue with one of the latest MLLMs: GPT-4.1 to understand why MLLMs fail to tell the time and whether fine-tuning can solve the problem. The results show how models are making progress in reading the time on analog clocks. But have they really learned to do it, or have they only learned patterns in their training datasets? In this work we put the models to the test with different clocks to illustrate the limitations of MLLMs to abstract and generalize.