mlsanityData to 7 Oct 2026

Fusing Perceptual Vision Experts with Multimodal Large Language Models for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation

arXiv 2608.24934 ↗First posted 23 Aug 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Agricultural vision-language models

Abstract (arXiv)

Accurate field plant disease diagnosis requires reliable fusion of uncertain and conflicting perceptual evidence. We present the Hybrid Hierarchical Multi-Agent Framework (H2MAF), combining decision-level fusion of EfficientNet-B3 and ConvNeXt-Tiny with semantic arbitration by open-weight multimodal large language models (MLLMs), Gemma 4 E4B and Qwen3.5 4B, using structured JSON evidence to generate explainable diagnoses, risk levels, treatment urgency, and financial exposure. (H2MAF) is evaluated on 14,364 images (1,370 test images) across PlantDoc (2,922 images, 27 classes) and two non-public, continuously captured Cornell robot-acquired field datasets: Stage 2 (20 GB; 4,215 images) and Stage 4 (40 GB; 7,227 images), covering Early Blight, Late Blight, and Septoria Leaf Spot under uncontrolled field conditions. On PlantDoc, Gemma improves accuracy from 63.9% to 68.5%, achieving +7.6 points on the 41.7% CNN-conflict subset. Cornell accuracies reach 99.3% and 98.9%, with only 1.7-4.1% disagreement, demonstrating conflict-dependent MLLM utility. The critical-risk error of gemma is 0.14-0.5 points, whereas Qwen overflags by 3.5-14.4 points. These results establish MLLM arbitration as a promising, yet calibration-dependent, approach for explainable agricultural AI and robotic field decision support. Github Link: https://github.com/Applied-AI-Research-Lab/Explainable-AI-Plant-Disease-Detection

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agricultural vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 58 references that are in the map. First posted on arXiv 23 Aug 2026.

Builds on

19 of 58 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  11. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  12. 2023A Survey on Multimodal Large Language Modelscited by 898
7 more
  1. 2016Using Deep Learning for Image-Based Plant Disease Detectioncited by 216
  2. 2018Deep learning in agriculture: A surveycited by 160
  3. 2016Grad-CAM: Why did you say that?cited by 159
  4. 2015An open access repository of images on plant health to enable the development of mobile disease diagnosticscited by 132
  5. 2019PlantDoc: A Dataset for Visual Plant Disease Detectioncited by 65
  6. 2025Plant Disease Detection through Multimodal Large Language Models and Convolutional Neural Networkscited by 3
  7. 2025Agentic AI with Orchestrator-Agent Trust: A Modular Visual Classification Framework with Trust-Aware Orchestration and RAG-Based Reasoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LeafNet: A Large-Scale Dataset and Comprehensive Benchmark for Foundational Vision-Language Understanding of Plant Diseases
  2. 2026SAGE: Scalable Agentic Grounded Evaluation for Crop Disease Diagnosis ↗
  3. 2026AgriChain Visually Grounded Expert Verified Reasoning for Interpretable Agricultural Vision Language Models
  4. 2026AgriChat: A Multimodal Large Language Model for Agriculture Image Understanding
  5. 2025A Multimodal Benchmark Dataset and Model for Crop Disease Diagnosis
  6. 2026A Multi-Modal Generative Model for Tomato Disease Leaves Understanding
  7. 2026Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry
  8. 2026Benchmarking Vision-Language Models for Microscopic Plant Image Understanding