mlsanityData to 7 Oct 2026

SAFE-Cascade: Cost-Adaptive Vision-Language Routing for Chart Question Answering

arXiv 2606.19646 ↗First posted 17 Jun 2026cs.IR cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Vision-language models (VLMs) are powerful for chart question answering, but invoking a VLM for every query can be unnecessarily expensive when many questions are answerable from OCR text and lightweight language reasoning. We demonstrate SAFE-Cascade, an interactive system for cost-adaptive chart question answering. Given a chart image and a natural-language question, SAFE-Cascade first extracts chart text with OCR, obtains a provisional answer from a text-only language model, and then uses a learned router to decide whether to accept the text answer or escalate to a VLM. The demo exposes this decision process to users: OCR evidence, text-only answer, routing probability, escalation decision, final answer, estimated cost, and estimated latency are shown side by side. SAFE-Cascade is designed as a transparent interface for understanding when visual grounding is actually needed. Users can upload or select charts, ask questions, inspect the evidence used by each pathway, compare text-only and VLM answers, and adjust the escalation threshold to explore the accuracy-cost frontier. The system is implemented with Azure Document Intelligence for OCR, gpt-5-mini as the text-only model, gemini-2.5-flash-image as the VLM, and a Random Forest router trained on inference-time features. On a held-out ChartQA test split of 375 examples from a 2,500-example experiment, SAFE-Cascade achieves 69.1% unified accuracy with 73.1% VLM invocation, compared with 67.7% accuracy and 100% VLM invocation for the full-VLM baseline. The observed +1.4 percentage-point difference is statistically uncertain, so we interpret SAFE-Cascade as matching full-VLM performance while reducing VLM calls by 26.9% and estimated cost by 9.3%. The demonstration shows how selective modality routing can make multimodal knowledge systems more transparent, tunable, and cost-aware.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 3 of its 9 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

3 of 9 references.

References · 3

  1. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  2. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  3. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustness
  2. 2025POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering
  3. 2024MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems
  4. 2025RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning
  5. 2025Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answering
  6. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering
  7. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering
  8. 2026Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models