mlsanityData to 14 Oct 2026

Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images

arXiv 2504.08727 ↗First posted 11 Apr 2025cs.CV cs.AI cs.CYcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

We present a system using Multimodal LLMs (MLLMs) to analyze a large database with tens of millions of images captured at different times, with the aim of discovering patterns in temporal changes. Specifically, we aim to capture frequent co-occurring changes ("trends") across a city over a certain period. Unlike previous visual analyses, our analysis answers open-ended queries (e.g., "what are the frequent types of changes in the city?") without any predetermined target subjects or training labels. These properties cast prior learning-based or unsupervised visual analysis tools unsuitable. We identify MLLMs as a novel tool for their open-ended semantic understanding capabilities. Yet, our datasets are four orders of magnitude too large for an MLLM to ingest as context. So we introduce a bottom-up procedure that decomposes the massive visual analysis problem into more tractable sub-problems. We carefully design MLLM-based solutions to each sub-problem. During experiments and ablation studies with our system, we find it significantly outperforms baselines and is able to discover interesting trends from images captured in large cities (e.g., "addition of outdoor dining,", "overpass was painted blue," etc.). See more results and interactive demos at https://boyangdeng.com/visual-chronicles.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 62 references that are in the map. First posted on arXiv 11 Apr 2025.

Builds on

35 of 62 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  9. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  10. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  11. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  12. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
23 more
  1. 2023Can large language models provide useful feedback on research papers? A large-scale empirical analysiscited by 200
  2. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  3. 2017Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the UScited by 66
  4. 2016Deep Learning the City : Quantifying Urban Perception At A Global Scalecited by 65
  5. 2013Recognizing Image Stylecited by 64
  6. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 62
  7. 2015A Century of Portraits: A Visual Historical Record of American High School Yearbookscited by 54
  8. 2023Describing Differences in Image Sets with Natural Languagecited by 53
  9. 2023Changes to Captions: An Attentive Network for Remote Sensing Change Captioningcited by 41
  10. 2022Image Difference Captioning with Pre-training and Contrastive Learningcited by 39
  11. 2024Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusioncited by 39
  12. 2015Transfer Learning from Deep Features for Remote Sensing and Poverty Mappingcited by 37
  13. 2019Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learningcited by 26
  14. 2024Explaining Datasets in Words: Statistical Models with Natural Language Parameterscited by 21
  15. 2017StreetStyle: Exploring world-wide clothing styles from millions of photoscited by 16
  16. 2022GSCLIP : A Framework for Explaining Distribution Shifts in Natural Languagecited by 14
  17. 2024Discover and Mitigate Multiple Biased Subgroups in Image Classifierscited by 11
  18. 2022What's in a Decade? Transforming Faces Through Timecited by 9
  19. 2023Prototype-based Dataset Comparisoncited by 9
  20. 2024Organizing Unstructured Image Collections using Natural Languagecited by 7
  21. 20153D Time-lapse Reconstruction from Internet Photoscited by 5
  22. 2024Diffusion Models as Data Mining Toolscited by 5
  23. 2024What could go wrong? Discovering and describing failure modes in computer visioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SCD4VPR: Multi-modal Scene Change Detection for Long-term Visual Place Recognition Database Updatecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding
  2. 2025MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis
  3. 2024Visualization Literacy of Multimodal Large Language Models: A Comparative Study
  4. 2024AdaptVision: Dynamic Input Scaling in MLLMs for Versatile Scene Understanding
  5. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding
  6. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs
  7. 2026MMTA: Benchmarking Multimodal Temporal Analysis with Time Series, Text, and Vision
  8. 2025Evaluating Graphical Perception with Multimodal LLMs