# AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment

arXiv: https://arxiv.org/abs/2410.01407 · HTML: https://mlsanity.com/paper/2410.01407/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Agricultural vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Agricultural vision-language models](https://mlsanity.com/mllm/agricultural-vision-language-models/)

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization](https://mlsanity.com/paper/2309.16020/) (2023)
- [No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance](https://mlsanity.com/paper/2404.04125/) (2024)
- [Text-To-Concept (and Back) via Cross-Model Alignment](https://mlsanity.com/paper/2305.06386/) (2023)
- [A Realistic Fish-Habitat Dataset to Evaluate Algorithms for Underwater Visual Analysis](https://mlsanity.com/paper/2008.12603/) (2020)
- [Leveraging Vision Language Models for Specialized Agricultural Tasks](https://mlsanity.com/paper/2407.19617/) (2024)
- [AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization](https://mlsanity.com/paper/2407.08156/) (2024)
- [PAPERCLIP: Associating Astronomical Observations and Natural Language with Multi-Modal Models](https://mlsanity.com/paper/2403.08851/) (2024)
- [Deep-Learning-based Counting Methods, Datasets, and Applications in Agriculture -- A Review](https://mlsanity.com/paper/2303.02632/) (2023)

## Built on by (most cited first, 13 of 13)

- [AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning](https://mlsanity.com/paper/2410.08405/) (2024)
- [AgriGPT-VL: Agricultural Vision-Language Understanding Suite](https://mlsanity.com/paper/2510.04002/) (2025)
- [AgriChat: A Multimodal Large Language Model for Agriculture Image Understanding](https://mlsanity.com/paper/2603.16934/) (2026)
- [When Language Model Guides Vision: Grounding DINO for Cattle Muzzle Detection](https://mlsanity.com/paper/2509.06427/) (2025)
- [TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases](https://mlsanity.com/paper/2608.08727/) (2026)
- [AI in Agriculture: A Survey of Deep Learning Techniques for Crops, Fisheries and Livestock](https://mlsanity.com/paper/2507.22101/) (2025)
- [An Efficient Remote Sensing Super Resolution Method Exploring Diffusion Priors and Multi-Modal Constraints for Crop Type Mapping](https://mlsanity.com/paper/2510.23382/) (2025)
- [DINOv3 Meets YOLO26 for Weed Detection in Vegetable Crops](https://mlsanity.com/paper/2603.00160/) (2026)
- [Geometry-Aware CLIP Retrieval via Local Cross-Modal Alignment and Steering](https://mlsanity.com/paper/2604.16487/) (2026)
- [CropVLM: A Domain-Adapted Vision-Language Model for Open-Set Crop Analysis](https://mlsanity.com/paper/2605.03259/) (2026)
- [AgriField-40K: Adapting Vision Models to Agriculture With Efficient Continual Pretraining](https://mlsanity.com/paper/2608.07984/) (2026)
- [You Are What You Prompt: Prompt Quality, Domain Shift, and Uncertainty in Agrifood Vision-Language Models](https://mlsanity.com/paper/2608.18116/) (2026)
- [A Multi-Modal Generative Model for Tomato Disease Leaves Understanding](https://mlsanity.com/paper/2609.19555/) (2026)

## Abstract (arXiv)

Capitalizing on vast amount of image-text data, large-scale vision-language pre-training has demonstrated remarkable zero-shot capabilities and has been utilized in several applications. However, models trained on general everyday web-crawled data often exhibit sub-optimal performance for specialized domains, likely due to domain shift. Recent works have tackled this problem for some domains (e.g., healthcare) by constructing domain-specialized image-text data. However, constructing a dedicated large-scale image-text dataset for sustainable area of agriculture and livestock is still open to research. Further, this domain desires fine-grained feature learning due to the subtle nature of the downstream tasks (e.g, nutrient deficiency detection, livestock breed classification). To address this we present AgriCLIP, a vision-language foundational model dedicated to the domain of agriculture and livestock. First, we propose a large-scale dataset, named ALive, that leverages customized prompt generation strategy to overcome the scarcity of expert annotations. Our ALive dataset covers crops, livestock, and fishery, with around 600,000 image-text pairs. Second, we propose a training pipeline that integrates both contrastive and self-supervised learning to learn both global semantic and local fine-grained domain-specialized features. Experiments on diverse set of 20 downstream tasks demonstrate the effectiveness of AgriCLIP framework, achieving an absolute gain of 7.8% in terms of average zero-shot classification accuracy, over the standard CLIP adaptation via domain-specialized ALive dataset. Our ALive dataset and code can be accessible at hrefhttps://github.com/umair1221/AgriCLIP/tree/mainGithub.