# Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images

arXiv: https://arxiv.org/abs/2504.08727 · HTML: https://mlsanity.com/paper/2504.08727/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 11 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 30 of 35)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US](https://mlsanity.com/paper/1702.06683/) (2017)
- [Deep Learning the City : Quantifying Urban Perception At A Global Scale](https://mlsanity.com/paper/1608.01769/) (2016)
- [Recognizing Image Style](https://mlsanity.com/paper/1311.3715/) (2013)
- [Goal Driven Discovery of Distributional Differences via Language Descriptions](https://mlsanity.com/paper/2302.14233/) (2023)
- [A Century of Portraits: A Visual Historical Record of American High School Yearbooks](https://mlsanity.com/paper/1511.02575/) (2015)
- [Describing Differences in Image Sets with Natural Language](https://mlsanity.com/paper/2312.02974/) (2023)
- [Changes to Captions: An Attentive Network for Remote Sensing Change Captioning](https://mlsanity.com/paper/2304.01091/) (2023)
- [Image Difference Captioning with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2202.04298/) (2022)
- [Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion](https://mlsanity.com/paper/2407.13759/) (2024)
- [Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping](https://mlsanity.com/paper/1510.00098/) (2015)
- [Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning](https://mlsanity.com/paper/1903.02678/) (2019)
- [Explaining Datasets in Words: Statistical Models with Natural Language Parameters](https://mlsanity.com/paper/2409.08466/) (2024)
- [StreetStyle: Exploring world-wide clothing styles from millions of photos](https://mlsanity.com/paper/1706.01869/) (2017)
- [GSCLIP : A Framework for Explaining Distribution Shifts in Natural Language](https://mlsanity.com/paper/2206.15007/) (2022)
- [Discover and Mitigate Multiple Biased Subgroups in Image Classifiers](https://mlsanity.com/paper/2403.12777/) (2024)
- [What's in a Decade? Transforming Faces Through Time](https://mlsanity.com/paper/2210.06642/) (2022)

## Built on by (most cited first, 1 of 1)

- [SCD4VPR: Multi-modal Scene Change Detection for Long-term Visual Place Recognition Database Update](https://mlsanity.com/paper/2604.11402/) (2026)

## Abstract (arXiv)

We present a system using Multimodal LLMs (MLLMs) to analyze a large database with tens of millions of images captured at different times, with the aim of discovering patterns in temporal changes. Specifically, we aim to capture frequent co-occurring changes ("trends") across a city over a certain period. Unlike previous visual analyses, our analysis answers open-ended queries (e.g., "what are the frequent types of changes in the city?") without any predetermined target subjects or training labels. These properties cast prior learning-based or unsupervised visual analysis tools unsuitable. We identify MLLMs as a novel tool for their open-ended semantic understanding capabilities. Yet, our datasets are four orders of magnitude too large for an MLLM to ingest as context. So we introduce a bottom-up procedure that decomposes the massive visual analysis problem into more tractable sub-problems. We carefully design MLLM-based solutions to each sub-problem. During experiments and ablation studies with our system, we find it significantly outperforms baselines and is able to discover interesting trends from images captured in large cities (e.g., "addition of outdoor dining,", "overpass was painted blue," etc.). See more results and interactive demos at https://boyangdeng.com/visual-chronicles.