mlsanityData to 7 Oct 2026

BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

arXiv 2608.17895 ↗First posted 18 Aug 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) have made significant strides in visual comprehension, their ability to reason about text-dense, professional documents remains incompletely evaluated. Existing benchmarks emphasize information extraction, require external domain knowledge, or cover professional documents only as one of many settings. They are also largely English- or Chinese-centric, leaving other languages and Russian, in particular, substantially underrepresented. To address these limitations, we introduce BEAR-Bench (Bilingual Enterprise and Academic Reasoning), a self-contained, complex English-and-Russian benchmark comprising 1000 human-annotated questions based on text-rich business and scientific documents. We evaluate 16 proprietary and open-weight MLLMs, including Gemini 3.1 Pro and Qwen3.5-397B, on BEAR-Bench and observe clear headroom even for the strongest systems. Finally, we use the resulting model outputs to compare existing hallucination detection methods, evaluating not only how often models fail on BEAR-Bench but also how reliably those failures can be identified.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 12 of its 41 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

12 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoning22 May
  2. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan

Other references · 10

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2024A Survey on LLM-as-a-Judgecited by 1,384
  5. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  6. 2026Gemma 4 Technical Reportcited by 187
  7. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 165
  8. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
  9. 2026FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answeringcited by 1
  10. 2026CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understandingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Measuring Epistemic Humility in Multimodal Large Language Models
  2. 2024MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation
  3. 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
  4. 2025HalluMix: A Task-Agnostic, Multi-Domain Benchmark for Real-World Hallucination Detection
  5. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Models
  6. 2025SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Models
  7. 2026Benchmarking Deflection and Hallucination in Large Vision-Language Models
  8. 2026MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents