mlsanityData to 7 Oct 2026

AgMMU: A Comprehensive Agricultural Multimodal Understanding Benchmark

arXiv 2504.10568 ↗First posted 14 Apr 2025cs.CVcited by 15

On the map: Multimodal LLMs / vision-language models → Agricultural vision-language models

Abstract (arXiv)

We present AgMMU, a challenging real-world benchmark for evaluating and advancing vision-language models (VLMs) in the knowledge-intensive domain of agriculture. Unlike prior datasets that rely on crowdsourced prompts, AgMMU is distilled from 116,231 authentic dialogues between everyday growers and USDA-authorized Cooperative Extension experts. Through a three-stage pipeline: automated knowledge extraction, QA generation, and human verification, we construct (i) AgMMU, an evaluation set of 746 multiple-choice questions (MCQs) and 746 open-ended questions (OEQs), and (ii) AgBase, a development corpus of 57,079 multimodal facts covering five high-stakes agricultural topics: insect identification, species identification, disease categorization, symptom description, and management instruction. Benchmarking 12 leading VLMs reveals pronounced gaps in fine-grained perception and factual grounding. Open-sourced models trail after proprietary ones by a wide margin. Simple fine-tuning on AgBase boosts open-sourced model performance on challenging OEQs for up to 11.6% on average, narrowing this gap and also motivating future research to propose better strategies in knowledge extraction and distillation from AgBase. We hope AgMMU stimulates research on domain-specific knowledge integration and trustworthy decision support in agriculture AI development.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Agricultural vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 56 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

29 of 56 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024AgriBench: A Hierarchical Agriculture Benchmark for Multimodal Large Language Models30 Nov
  2. 2024AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning10 Oct
  3. 2024VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images28 Aug

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  10. 2019Towards VQA Models That Can Readcited by 1,696
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  12. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
14 more
  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  2. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  3. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  4. 2024Measuring short-form factuality in large language modelscited by 345
  5. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 159
  6. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  7. 2024RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculturecited by 80
  8. 2019PlantDoc: A Dataset for Visual Plant Disease Detectioncited by 65
  9. 2023A Step Towards Worldwide Biodiversity Assessment: The BIOSCAN-1M Insect Datasetcited by 30
  10. 2024Benchmarking In-the-wild Multimodal Disease Recognition and A Versatile Baselinecited by 19
  11. 2025A Multimodal Benchmark Dataset and Model for Crop Disease Diagnosiscited by 14
  12. 2023GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Modelscited by 12
  13. 2023Deep learning powered real-time identification of insects using citizen science datacited by 8
  14. 2024Generating Diverse Agricultural Data for Vision-Based Farming Applicationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

By area:

  1. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  2. 2025MIRAGE: A Benchmark for Multimodal Information-Seeking and Reasoning in Agricultural Expert-Guided Conversationscited by 12
  3. 2025Towards Large Reasoning Models for Agriculturecited by 3
  4. 2025PlantExpertVQA: A Visual Question Answering Dataset for Benchmarking Vision-Language Models in Plant Sciencecited by 3
  5. 2026AgriChat: A Multimodal Large Language Model for Agriculture Image Understandingcited by 2
  6. 2025Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domaincited by 1
  7. 2025Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQAcited by 1
  8. 2026Agri-R1: Agricultural Reasoning for Disease Diagnosis via Automated-Synthesis and Reinforcement Learningcited by 1
  9. 2026LUCAS-MEGA: A Large-Scale Multimodal Dataset for Representation Learning in Soil-Environment Systemscited by 0
  10. 2026AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Groundingcited by 0
  11. 2026You Are What You Prompt: Prompt Quality, Domain Shift, and Uncertainty in Agrifood Vision-Language Modelscited by 0
  12. 2026From Vision to Harvest: Benchmarking Vision-Language Models for Multi-Arm Robotic Fruit Harvestingcited by 0
3 more
  1. 2026A Multi-Modal Generative Model for Tomato Disease Leaves Understandingcited by 0
  2. 2026AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Imagescited by 0
  3. 2026BananaVLM: A Domain-Adapted Vision Language Model for Banana Crop Disease Diagnosiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AgriGPT-VL: Agricultural Vision-Language Understanding Suite
  2. 2026AgroOmni: A Large-Scale Multi-view Agricultural Dataset for Cross-Scale Multimodal Reasoning
  3. 2026AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture
  4. 2026Vision-language models know more about agriculture than they show and rubric-grounded verifications close the gap
  5. 2024Agri-LLaVA: Knowledge-Infused Large Multimodal Assistant on Agricultural Pests and Diseases
  6. 2025AgroCoT: A Chain-of-Thought Benchmark for Evaluating Reasoning in Vision-Language Models for Agriculture
  7. 2025Are vision-language models ready to zero-shot replace supervised classification models in agriculture?
  8. 2025AgriDoctor: A Multimodal Intelligent Assistant for Agriculture