mlsanityData to 7 Oct 2026

SnapBench: Benchmarking Snap-and-Ask Multimodal Retrieval for Mobile Interactions

arXiv 2608.29607 ↗First posted 30 Aug 2026cs.CV cs.IRcited by 1

Not on the map. Why?

Abstract (arXiv)

Mobile AI acts as a visual oracle, empowering users to snap a picture of something and ask for information. Snap-and-ask retrieval is now one of the most common entry points for mobile AI, yet photos are often blurry, while text questions may be short or mistyped. Existing benchmarks only test on clean inputs or do not isolate paired robustness in snap-and-ask retrieval. Therefore, we introduce SnapBench, the first paired benchmark for robust snap-and-ask multimodal retrieval, spanning 1,145 queries, 9,085 gallery items under 53 controlled corruption conditions with human annotations. We evaluate 16 multimodal retrievers, covering dual-tower encoders and embedding-based VLMs. Results show that image corruptions substantially degrade retrieval, while text corruptions mainly affect text-only retrieval and have limited impact on joint retrieval. Clean image-only retrieval often outperforms joint retrieval, indicating the coarse-text drag and the lack of cross-modal fallback under noisy inputs. SnapBench provides a controlled testbed for evaluating robust retrieval in snap-and-ask scenarios. We further propose MOOR (Modality-anchored, Outlier-aware, Optimal Reweighting), a simple adaptive fusion approach, highlighting the need for reliability-aware modality calibration in snap-and-ask retrieval.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

30 of 51 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  4. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  5. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  6. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  7. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  8. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  9. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  10. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  11. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  12. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
18 more
  1. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  2. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  3. 2023Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entitiescited by 112
  4. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  5. 2023GeneCIS: A Benchmark for General Conditional Image Similaritycited by 46
  6. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddingscited by 37
  7. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillationcited by 31
  8. 2025GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environmentscited by 18
  9. 2026GameVerse: Can Vision-Language Models Learn from Video-based Reflection?cited by 7
  10. 2025Handling Label Noise via Instance-Level Difficulty Modeling and Dynamic Optimizationcited by 5
  11. 2025Mixture of Length and Pruning Experts for Knowledge Graphs Reasoningcited by 5
  12. 2025CapGeo-Bench: Decoupling Visual Perception from Reasoning and Evaluating Geometric Understandingcited by 4
  13. 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
  14. 2026IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streamscited by 4
  15. 2026LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledgercited by 3
  16. 2025Dynamic Residual Encoding with Slide-Level Contrastive Learning for End-to-End Whole Slide Image Representationcited by 2
  17. 2026Towards Spatial Supersensing in the Wildcited by 2
  18. 2026EviRank: Structured Relevance Evidence for Multimodal Image Re-rankingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning Multimodal Embeddings with Evidence-Aligned Readoutcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models
  2. 2024Benchmark Granularity and Model Robustness for Image-Text Retrieval
  3. 2026CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation
  4. 2026ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge
  5. 2024INQUIRE: A Natural World Text-to-Image Retrieval Benchmark
  6. 2026Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation
  7. 2025MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval
  8. 2026Reason Before You Retrieve: Agentic Planning for Multi-modal RAG