# "Does the cafe entrance look accessible? Where is the door?" Towards Geospatial AI Agents for Visual Inquiries

arXiv: https://arxiv.org/abs/2508.15752 · HTML: https://mlsanity.com/paper/2508.15752/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 67 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 17 of 17)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs](https://mlsanity.com/paper/2404.07449/) (2024)
- [GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Reality](https://mlsanity.com/paper/2404.08213/) (2024)
- [GIScience in the Era of Artificial Intelligence: A Research Agenda Towards Autonomous GIS](https://mlsanity.com/paper/2503.23633/) (2025)
- [CitySurfaces: City-Scale Semantic Segmentation of Sidewalk Materials](https://mlsanity.com/paper/2201.02260/) (2022)
- [StreetNav: Leveraging Street Cameras to Support Precise Outdoor Navigation for Blind Pedestrians](https://mlsanity.com/paper/2310.00491/) (2023)
- [Visual Question Answering on Multiple Remote Sensing Image Modalities](https://mlsanity.com/paper/2505.15401/) (2025)
- [CookAR: Affordance Augmentations in Wearable AR to Support Kitchen Tool Interactions for People with Low Vision](https://mlsanity.com/paper/2407.13515/) (2024)
- [StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI](https://mlsanity.com/paper/2508.08524/) (2025)
- [RampNet: A Two-Stage Pipeline for Bootstrapping Curb Ramp Detection in Streetscape Images from Open Government Metadata](https://mlsanity.com/paper/2508.09415/) (2025)
- [Automatic Map Update Using Dashcam Videos](https://mlsanity.com/paper/2109.12131/) (2021)
- [RASSAR: Room Accessibility and Safety Scanning in Augmented Reality](https://mlsanity.com/paper/2404.07479/) (2024)
- [Accessibility Scout: Personalized Accessibility Scans of Built Environments](https://mlsanity.com/paper/2507.23190/) (2025)
- [Quantifying the presence of graffiti in urban environments](https://mlsanity.com/paper/1904.04336/) (2019)
- [Towards Data-Informed Interventions: Opportunities and Challenges of Street-level Multimodal Sensing](https://mlsanity.com/paper/2410.22092/) (2024)

## Abstract (arXiv)

Interactive digital maps have revolutionized how people travel and learn about the world; however, they rely on pre-existing structured data in GIS databases (e.g., road networks, POI indices), limiting their ability to address geo-visual questions related to what the world looks like. We introduce our vision for Geo-Visual Agents--multimodal AI agents capable of understanding and responding to nuanced visual-spatial inquiries about the world by analyzing large-scale repositories of geospatial images, including streetscapes (e.g., Google Street View), place-based photos (e.g., TripAdvisor, Yelp), and aerial imagery (e.g., satellite photos) combined with traditional GIS data sources. We define our vision, describe sensing and interaction approaches, provide three exemplars, and enumerate key challenges and opportunities for future work.