# Snap and Diagnose: An Advanced Multimodal Retrieval System for Identifying Plant Diseases in the Wild

arXiv: https://arxiv.org/abs/2408.14723 · HTML: https://mlsanity.com/paper/2408.14723/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 13 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 2 of 2)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An open access repository of images on plant health to enable the development of mobile disease diagnostics](https://mlsanity.com/paper/1511.08060/) (2015)

## Built on by (most cited first, 12 of 12)

- [PlantSeg: A Large-Scale In-the-wild Dataset for Plant Disease Segmentation](https://mlsanity.com/paper/2409.04038/) (2024)
- [Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptation](https://mlsanity.com/paper/2510.09867/) (2025)
- [CF-PRNet: Coarse-to-Fine Prototype Refining Network for Point Cloud Completion and Reconstruction](https://mlsanity.com/paper/2409.08443/) (2024)
- [Track Any Peppers: Weakly Supervised Sweet Pepper Tracking Using VLMs](https://mlsanity.com/paper/2411.06702/) (2024)
- [Plant Disease Detection through Multimodal Large Language Models and Convolutional Neural Networks](https://mlsanity.com/paper/2504.20419/) (2025)
- [Augment to Segment: Tackling Pixel-Level Imbalance in Wheat Disease and Pest Segmentation](https://mlsanity.com/paper/2509.09961/) (2025)
- [Distributed Zero-Shot Learning for Visual Recognition](https://mlsanity.com/paper/2511.08170/) (2025)
- [Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQA](https://mlsanity.com/paper/2509.24350/) (2025)
- [StomataSeg: Semi-Supervised Instance Segmentation for Sorghum Stomatal Components](https://mlsanity.com/paper/2602.00703/) (2026)
- [CropVLM: A Domain-Adapted Vision-Language Model for Open-Set Crop Analysis](https://mlsanity.com/paper/2605.03259/) (2026)
- [Hallucination Behavior in Multimodal LLMs Across Agricultural Image Interpretation and Generation Tasks](https://mlsanity.com/paper/2605.27595/) (2026)
- [Benchmarking Vision-Language Models for Microscopic Plant Image Understanding](https://mlsanity.com/paper/2606.22497/) (2026)

## Abstract (arXiv)

Plant disease recognition is a critical task that ensures crop health and mitigates the damage caused by diseases. A handy tool that enables farmers to receive a diagnosis based on query pictures or the text description of suspicious plants is in high demand for initiating treatment before potential diseases spread further. In this paper, we develop a multimodal plant disease image retrieval system to support disease search based on either image or text prompts. Specifically, we utilize the largest in-the-wild plant disease dataset PlantWild, which includes over 18,000 images across 89 categories, to provide a comprehensive view of potential diseases relating to the query. Furthermore, cross-modal retrieval is achieved in the developed system, facilitated by a novel CLIP-based vision-language model that encodes both disease descriptions and disease images into the same latent space. Built on top of the retriever, our retrieval system allows users to upload either plant disease images or disease descriptions to retrieve the corresponding images with similar characteristics from the disease dataset to suggest candidate diseases for end users' consideration.