mlsanityData to 14 Oct 2026

Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?

arXiv 2609.17882 ↗First posted 15 Sep 2026cs.CV cs.CY cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

An important component of urban accessibility, particularly for wheelchair users and people with reduced mobility, is sidewalk compliance with measurable requirements. We test whether effective width, longitudinal slope, cross slope, and pavement condition can be assessed reliably from pedestrian-level imagery using vision-language models (VLMs). We present the first application of sampling-based conformal prediction (CP) for VLM-based accessibility assessment. We evaluate four VLMs on 514 sidewalk images from Seoul, South Korea, with field-measured ground truth. Conformal calibration attains the nominal 90% coverage for all models and attributes, but the calibrated regions differ in informativeness. Effective width yields the most informative estimates, with a mean interval half-width of about 1.0 m for the best model. Since every model overestimates width, asymmetric calibration shortens the intervals by up to 33% at unchanged coverage. Longitudinal slope is marginally informative, cross-slope intervals are too wide to resolve regulatory thresholds, and pavement-condition sets degenerate to all five grades (A-E) for three of the four models. Uncalibrated intervals from raw sampling dispersion cover only 17-47% of field-measured values at a nominal 90% level. Among the images with the most self-consistent responses, these intervals miss the field-measured value in up to 96% of cases. Response self-consistency is therefore not evidence of accuracy, and sampling dispersion cannot be interpreted as uncertainty until it has been calibrated against field-measured ground truth. No quantitative attribute reaches the precision required for general compliance assessment, but CP identifies from calibration data alone which attributes can support screening of segments far from the thresholds. We release the annotated pedestrian-level images and their corresponding field-measured attribute values.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 87 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

38 of 87 references.

References · 38

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  5. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  6. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  7. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  8. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  9. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  10. 2019Conformalized Quantile Regressioncited by 629
  11. 2020Classification with Valid and Adaptive Coveragecited by 396
  12. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 272
26 more
  1. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  2. 2023Conformal Language Modelingcited by 170
  3. 2024Language Models with Conformal Factuality Guaranteescited by 131
  4. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  5. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  6. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 73
  7. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 69
  8. 2024Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Modelscited by 54
  9. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 53
  10. 2024Uncertainty-Aware Evaluation for Vision-Language Modelscited by 29
  11. 2024Conformal Prediction: A Data Perspectivecited by 25
  12. 2025SConU: Selective Conformal Uncertainty in Large Language Modelscited by 23
  13. 2025Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Modelscited by 21
  14. 2025Too Consistent to Detect: A Study of Self-Consistent Errors in LLMscited by 18
  15. 2025Conformal Prediction for Zero-Shot Modelscited by 15
  16. 2024Are foundation models for computer vision good conformal predictors?cited by 13
  17. 2025Full Conformal Adaptation of Medical Vision-Language Modelscited by 9
  18. 2025LLMs are Overconfident: Evaluating Confidence Interval Calibration with FermiEvalcited by 8
  19. 2025Streetscape Analysis with Generative AI (SAGAI): Vision-Language Assessment and Mapping of Urban Scenescited by 6
  20. 2025Domain-Shift-Aware Conformal Prediction for Large Language Modelscited by 6
  21. 2025Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Predictioncited by 3
  22. 2025Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guaranteescited by 2
  23. 2025Can a Large Language Model Assess Urban Design Quality? Evaluating Walkability Metrics Across Expertise Levelscited by 1
  24. 2026CONRep: Uncertainty-Aware Vision-Language Report Drafting Using Conformal Predictioncited by 1
  25. 2026Towards Human-AI Accessibility Mapping in India: VLM-Guided Annotations and POI-Centric Analysis in Chandigarhcited by 1
  26. 2026Do VLMs See What Sensors Feel? A Scalable Expert-Guided Design for Wheelchair Accessibility Assessment from Street Viewcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Street View Imagery to Street Quality Indicators: Vision Language Inference for the Suburban 15-minute City
  2. 2026A Vision-Language Framework for Measuring Social Life on Sidewalks
  3. 2026Benchmarks for Vision-Language Models in Urban Perception Should Be Reliability-Aware and Negotiated
  4. 2025Do Vision-Language Models See Urban Scenes as People Do? An Urban Perception Benchmark
  5. 2025Interpretable Multimodal Framework for Human-Centered Street Assessment: Integrating Visual-Language Models for Perceptual Urban Diagnostics
  6. 2025Towards General Urban Monitoring with Vision-Language Models: A Review, Evaluation, and a Research Agenda
  7. 2026MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps ?
  8. 2026You Cannot Photograph the Same Street Twice: Reliability Limits in Vision-Language Measurement of Urban Change