# Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoning

arXiv: https://arxiv.org/abs/2307.06166 · HTML: https://mlsanity.com/paper/2307.06166/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 12 Jul 2023.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [FLAVA: A Foundational Language And Vision Alignment Model](https://mlsanity.com/paper/2112.04482/) (2021)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model](https://mlsanity.com/paper/2304.15010/) (2023)
- [SLIP: Self-supervision meets Language-Image Pre-training](https://mlsanity.com/paper/2112.12750/) (2021)
- [Unified Contrastive Learning in Image-Text-Label Space](https://mlsanity.com/paper/2204.03610/) (2022)
- [VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling](https://mlsanity.com/paper/2111.12681/) (2021)
- [Align and Prompt: Video-and-Language Pre-training with Entity Prompts](https://mlsanity.com/paper/2112.09583/) (2021)
- [PlaNet - Photo Geolocation with Convolutional Neural Networks](https://mlsanity.com/paper/1602.05314/) (2016)
- [Localizing and Orienting Street Views Using Overhead Imagery](https://mlsanity.com/paper/1608.00161/) (2016)
- [CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps](https://mlsanity.com/paper/1808.02130/) (2018)
- [There is a Time and Place for Reasoning Beyond the Image](https://mlsanity.com/paper/2203.00758/) (2022)

## Built on by (most cited first, 22 of 22)

- [Evaluating Precise Geolocation Inference Capabilities of Vision Language Models](https://mlsanity.com/paper/2502.14412/) (2025)
- [NAVIG: Natural Language-guided Analysis with Vision Language Models for Image Geo-localization](https://mlsanity.com/paper/2502.14638/) (2025)
- [ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance](https://mlsanity.com/paper/2505.18757/) (2025)
- [Multi-modal Summarization in Model-Based Engineering: Automotive Software Development Case Study](https://mlsanity.com/paper/2503.04506/) (2025)
- [AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction](https://mlsanity.com/paper/2506.18472/) (2025)
- [Beyond Captioning: Task-Specific Prompting for Improved VLM Performance in Mathematical Reasoning](https://mlsanity.com/paper/2410.05928/) (2024)
- [GeoShield: Safeguarding Geolocation Privacy from Vision-Language Models via Adversarial Perturbations](https://mlsanity.com/paper/2508.03209/) (2025)
- [Watch Before You Answer: Learning from Visually Grounded Post-Training](https://mlsanity.com/paper/2604.05117/) (2026)
- [V-CECE: Visual Counterfactual Explanations via Conceptual Edits](https://mlsanity.com/paper/2509.16567/) (2025)
- [CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captions](https://mlsanity.com/paper/2507.06210/) (2025)
- [Automatic benchmarking of large multimodal models via iterative experiment programming](https://mlsanity.com/paper/2406.12321/) (2024)
- [PuzzleGPT: Emulating Human Puzzle-Solving Ability for Time and Location Prediction](https://mlsanity.com/paper/2501.14210/) (2025)
- [BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusion](https://mlsanity.com/paper/2601.03713/) (2026)
- [GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM](https://mlsanity.com/paper/2407.10870/) (2024)
- [Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views](https://mlsanity.com/paper/2506.03371/) (2025)
- [Augmented Vision-Language Models: A Systematic Review](https://mlsanity.com/paper/2507.22933/) (2025)
- [Beyond Memorization: A Multi-Modal Ordinal Regression Benchmark to Expose Popularity Bias in Vision-Language Models](https://mlsanity.com/paper/2512.21337/) (2025)
- [LocationAgent: A Hierarchical Agent for Image Geolocation via Decoupling Strategy and Evidence from Parametric Knowledge](https://mlsanity.com/paper/2601.19155/) (2026)
- [Constructive Apraxia: An Unexpected Limit of Instructible Vision-Language Models and Analog for Human Cognitive Disorders](https://mlsanity.com/paper/2410.03551/) (2024)
- [Beyond the Linear Separability Ceiling: Aligning Representations in VLMs](https://mlsanity.com/paper/2507.07574/) (2025)
- [Spatio-Temporal Data Enhanced Vision-Language Model for Traffic Scene Understanding](https://mlsanity.com/paper/2511.08978/) (2025)
- [NewsRECON: News Article Retrieval for Image Contextualization](https://mlsanity.com/paper/2601.14121/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) are expected to be capable of reasoning with commonsense knowledge as human beings. One example is that humans can reason where and when an image is taken based on their knowledge. This makes us wonder if, based on visual cues, Vision-Language Models that are pre-trained with large-scale image-text resources can achieve and even outperform human's capability in reasoning times and location. To address this question, we propose a two-stage recognitionspace and reasoningspace probing task, applied to discriminative and generative VLMs to uncover whether VLMs can recognize times and location-relevant features and further reason about it. To facilitate the investigation, we introduce WikiTiLo, a well-curated image dataset compromising images with rich socio-cultural cues. In the extensive experimental studies, we find that although VLMs can effectively retain relevant features in visual encoders, they still fail to make perfect reasoning. We will release our dataset and codes to facilitate future studies.