# You Cannot Photograph the Same Street Twice: Reliability Limits in Vision-Language Measurement of Urban Change

arXiv: https://arxiv.org/abs/2609.00649 · HTML: https://mlsanity.com/paper/2609.00649/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 7 of 7)

- [Deep Learning the City : Quantifying Urban Perception At A Global Scale](https://mlsanity.com/paper/1608.01769/) (2016)
- [Coverage and Bias of Street View Imagery in Mapping the Urban Environment](https://mlsanity.com/paper/2409.15386/) (2024)
- [CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series](https://mlsanity.com/paper/2401.01107/) (2024)
- [UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective](https://mlsanity.com/paper/2509.22228/) (2025)
- [Artifacts of Idiosyncracy in Global Street View Data](https://mlsanity.com/paper/2505.11046/) (2025)
- [Benchmarks for Vision-Language Models in Urban Perception Should Be Reliability-Aware and Negotiated](https://mlsanity.com/paper/2606.00871/) (2026)
- [PairWise Image Finder: An Open-source Tool for Finding Visually Aligned Street-Level Image Pairs for Urban Perception Studies](https://mlsanity.com/paper/2606.08795/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) are increasingly used to measure urban change from repeated street-level imagery, and the results are often mapped for individual sample points. Such maps assume that a perception score changes only if the street does. We test this assumption with repeated Google Street View captures of unchanged streets in five US cities and with controlled experiments that hold the photograph, scene or camera fixed. Re-photographing an unchanged street shifts its perception score by two-thirds of the average difference between two streets in the same city. Much of this shift arises in the scoring and averages out over question orders; the rest comes from the photographs and is not predicted by image statistics of weather and light. A single image measures a place moderately well, but the change at one location barely separates redeveloped from unchanged streets. All models tested score degraded images as worse-looking streets, and in crowdsourced imagery camera differences cause false reports of physical change unless both images are rendered through a common virtual camera. After aggregation, redeveloped streets are scored as wealthier and better maintained, with more enclosure and less greenery. Streets photographed in the same capture campaign share part of their error, which averaging within a district does not remove. For urban research and planning, VLM perception scores can support comparisons between groups of streets, such as redeveloped and unchanged streets photographed in the same campaigns, but not the identification of individual streets that improved or declined.