# MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversations

arXiv: https://arxiv.org/abs/2506.20100 · HTML: https://mlsanity.com/paper/2506.20100/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 27 of its 75 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 27 of 27)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues](https://mlsanity.com/paper/2402.14762/) (2024)
- [Phi-4-reasoning Technical Report](https://mlsanity.com/paper/2504.21318/) (2025)
- [An open access repository of images on plant health to enable the development of mobile disease diagnostics](https://mlsanity.com/paper/1511.08060/) (2015)
- [A Critical Evaluation of Evaluations for Long-form Question Answering](https://mlsanity.com/paper/2305.18201/) (2023)
- [MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation](https://mlsanity.com/paper/2104.00773/) (2021)
- [From Pixels to Prose: A Large Dataset of Dense Image Captions](https://mlsanity.com/paper/2406.10328/) (2024)
- [ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models](https://mlsanity.com/paper/2403.20194/) (2024)
- [Long-Form Answers to Visual Questions from Blind and Low Vision People](https://mlsanity.com/paper/2408.06303/) (2024)
- [Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation](https://mlsanity.com/paper/2402.13408/) (2024)
- [AgMMU: A Comprehensive Agricultural Multimodal Understanding Benchmark](https://mlsanity.com/paper/2504.10568/) (2025)
- [Agri-LLaVA: Knowledge-Infused Large Multimodal Assistant on Agricultural Pests and Diseases](https://mlsanity.com/paper/2412.02158/) (2024)
- [Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning](https://mlsanity.com/paper/2406.03600/) (2024)
- [Deep learning powered real-time identification of insects using citizen science data](https://mlsanity.com/paper/2306.02507/) (2023)
- [MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines](https://mlsanity.com/paper/1907.01669/) (2019)

## Built on by (most cited first, 13 of 13)

- [A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges](https://mlsanity.com/paper/2508.05668/) (2025)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Drift No More? Context Equilibria in Multi-Turn LLM Interactions](https://mlsanity.com/paper/2510.07777/) (2025)
- [A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications](https://mlsanity.com/paper/2510.16724/) (2025)
- [CPJ: Explainable Agricultural Pest Diagnosis via Caption-Prompt-Judge with LLM-Judged Refinement](https://mlsanity.com/paper/2512.24947/) (2025)
- [Multi-label Instance-level Generalised Visual Grounding in Agriculture](https://mlsanity.com/paper/2603.06699/) (2026)
- [AgroCoT: A Chain-of-Thought Benchmark for Evaluating Reasoning in Vision-Language Models for Agriculture](https://mlsanity.com/paper/2511.23253/) (2025)
- [AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification](https://mlsanity.com/paper/2603.13354/) (2026)
- [AMIGO: Agentic Multi-Image Grounding Oracle Benchmark](https://mlsanity.com/paper/2603.28662/) (2026)
- [Agri-CPJ: A Training-Free Explainable Framework for Agricultural Pest Diagnosis Using Caption-Prompt-Judge and LLM-as-a-Judge](https://mlsanity.com/paper/2604.23701/) (2026)
- [Benchmarking Vision-Language Models for Microscopic Plant Image Understanding](https://mlsanity.com/paper/2606.22497/) (2026)
- [Routing by Reasoning Need: Trajectory-Aware Decoding Control for Diffusion Vision-Language Models](https://mlsanity.com/paper/2609.11315/) (2026)
- [AgroGround: Multi-Granularity Grounded Recognition in Agriculture](https://mlsanity.com/paper/2610.04425/) (2026)

## Abstract (arXiv)

We introduce MIRAGE, a new benchmark for multimodal expert-level reasoning and decision-making in consultative interaction settings. Designed for the agriculture domain, MIRAGE captures the full complexity of expert consultations by combining natural user queries, expert-authored responses, and image-based context, offering a high-fidelity benchmark for evaluating models on grounded reasoning, clarification strategies, and long-form generation in a real-world, knowledge-intensive domain. Grounded in over 35,000 real user-expert interactions and curated through a carefully designed multi-step pipeline, MIRAGE spans diverse crop health, pest diagnosis, and crop management scenarios. The benchmark includes more than 7,000 unique biological entities, covering plant species, pests, and diseases, making it one of the most taxonomically diverse benchmarks available for vision-language models, grounded in the real world. Unlike existing benchmarks that rely on well-specified user inputs and closed-set taxonomies, MIRAGE features underspecified, context-rich scenarios with open-world settings, requiring models to infer latent knowledge gaps, handle rare entities, and either proactively guide the interaction or respond. Project Page: https://mirage-benchmark.github.io