# AgMMU: A Comprehensive Agricultural Multimodal Understanding Benchmark

arXiv: https://arxiv.org/abs/2504.10568 · HTML: https://mlsanity.com/paper/2504.10568/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Agricultural vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Agricultural vision-language models](https://mlsanity.com/mllm/agricultural-vision-language-models/)

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture](https://mlsanity.com/paper/2401.08406/) (2024)
- [PlantDoc: A Dataset for Visual Plant Disease Detection](https://mlsanity.com/paper/1911.10317/) (2019)
- [A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Dataset](https://mlsanity.com/paper/2307.10455/) (2023)
- [AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning](https://mlsanity.com/paper/2410.08405/) (2024)
- [Benchmarking In-the-wild Multimodal Disease Recognition and A Versatile Baseline](https://mlsanity.com/paper/2408.03120/) (2024)
- [A Multimodal Benchmark Dataset and Model for Crop Disease Diagnosis](https://mlsanity.com/paper/2503.06973/) (2025)
- [GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Models](https://mlsanity.com/paper/2310.06225/) (2023)
- [VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images](https://mlsanity.com/paper/2408.16176/) (2024)
- [Deep learning powered real-time identification of insects using citizen science data](https://mlsanity.com/paper/2306.02507/) (2023)
- [Generating Diverse Agricultural Data for Vision-Based Farming Applications](https://mlsanity.com/paper/2403.18351/) (2024)
- [AgriBench: A Hierarchical Agriculture Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2412.00465/) (2024)

## Built on by (most cited first, 16 of 16)

- [Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models](https://mlsanity.com/paper/2505.04921/) (2025)
- [MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversations](https://mlsanity.com/paper/2506.20100/) (2025)
- [Towards Large Reasoning Models for Agriculture](https://mlsanity.com/paper/2505.19259/) (2025)
- [PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Science](https://mlsanity.com/paper/2508.17117/) (2025)
- [AgriChat: A Multimodal Large Language Model for Agriculture Image Understanding](https://mlsanity.com/paper/2603.16934/) (2026)
- [Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domain](https://mlsanity.com/paper/2507.16974/) (2025)
- [Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQA](https://mlsanity.com/paper/2509.24350/) (2025)
- [Agri-R1: Agricultural Reasoning for Disease Diagnosis via Automated-Synthesis and Reinforcement Learning](https://mlsanity.com/paper/2601.04672/) (2026)
- [AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding](https://mlsanity.com/paper/2605.22034/) (2026)
- [LUCAS-MEGA: A Large-Scale Multimodal Dataset for Representation Learning in Soil-Environment Systems](https://mlsanity.com/paper/2605.04323/) (2026)
- [You Are What You Prompt: Prompt Quality, Domain Shift, and Uncertainty in Agrifood Vision-Language Models](https://mlsanity.com/paper/2608.18116/) (2026)
- [From Vision to Harvest: Benchmarking Vision-Language Models for Multi-Arm Robotic Fruit Harvesting](https://mlsanity.com/paper/2609.13606/) (2026)
- [A Multi-Modal Generative Model for Tomato Disease Leaves Understanding](https://mlsanity.com/paper/2609.19555/) (2026)
- [AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Images](https://mlsanity.com/paper/2609.20325/) (2026)
- [BananaVLM: A Domain-Adapted Vision Language Model for Banana Crop Disease Diagnosis](https://mlsanity.com/paper/2609.25040/) (2026)
- [AgroGround: Multi-Granularity Grounded Recognition in Agriculture](https://mlsanity.com/paper/2610.04425/) (2026)

## Abstract (arXiv)

We present AgMMU, a challenging real-world benchmark for evaluating and advancing vision-language models (VLMs) in the knowledge-intensive domain of agriculture. Unlike prior datasets that rely on crowdsourced prompts, AgMMU is distilled from 116,231 authentic dialogues between everyday growers and USDA-authorized Cooperative Extension experts. Through a three-stage pipeline: automated knowledge extraction, QA generation, and human verification, we construct (i) AgMMU, an evaluation set of 746 multiple-choice questions (MCQs) and 746 open-ended questions (OEQs), and (ii) AgBase, a development corpus of 57,079 multimodal facts covering five high-stakes agricultural topics: insect identification, species identification, disease categorization, symptom description, and management instruction. Benchmarking 12 leading VLMs reveals pronounced gaps in fine-grained perception and factual grounding. Open-sourced models trail after proprietary ones by a wide margin. Simple fine-tuning on AgBase boosts open-sourced model performance on challenging OEQs for up to 11.6% on average, narrowing this gap and also motivating future research to propose better strategies in knowledge extraction and distillation from AgBase. We hope AgMMU stimulates research on domain-specific knowledge integration and trustworthy decision support in agriculture AI development.