# Can SAM recognize crops? Quantifying the zero-shot performance of a semantic segmentation foundation model on generating crop-type maps using satellite imagery for precision agriculture

arXiv: https://arxiv.org/abs/2311.15138 · HTML: https://mlsanity.com/paper/2311.15138/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 25 Nov 2023.

## Builds on (most cited first, 6 of 6)

- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model](https://mlsanity.com/paper/2305.02034/) (2023)
- [Segment Anything in Non-Euclidean Domains: Challenges and Opportunities](https://mlsanity.com/paper/2304.11595/) (2023)
- [CalCROP21: A Georeferenced multi-spectral dataset of Satellite Imagery and Crop Labels](https://mlsanity.com/paper/2107.12499/) (2021)
- [Attention-augmented Spatio-Temporal Segmentation for Land Cover Mapping](https://mlsanity.com/paper/2105.02963/) (2021)
- [Spatio-temporal Crop Classification On Volumetric Data](https://mlsanity.com/paper/2103.10050/) (2021)

## Built on by (most cited first, 3 of 3)

- [Benchmarking Geospatial Foundation Models for Agriculture Applications](https://mlsanity.com/paper/2606.29664/) (2026)
- [Farm-Level, In-Season Crop Identification for India](https://mlsanity.com/paper/2507.02972/) (2025)
- [JEDI: JEPA-to-Edge Distillation for Efficient Cropland Segmentation from Satellite Imagery](https://mlsanity.com/paper/2609.07915/) (2026)

## Abstract (arXiv)

Climate change is increasingly disrupting worldwide agriculture, making global food production less reliable. To tackle the growing challenges in feeding the planet, cutting-edge management strategies, such as precision agriculture, empower farmers and decision-makers with rich and actionable information to increase the efficiency and sustainability of their farming practices. Crop-type maps are key information for decision-support tools but are challenging and costly to generate. We investigate the capabilities of Meta AI's Segment Anything Model (SAM) for crop-map prediction task, acknowledging its recent successes at zero-shot image segmentation. However, SAM being limited to up-to 3 channel inputs and its zero-shot usage being class-agnostic in nature pose unique challenges in using it directly for crop-type mapping. We propose using clustering consensus metrics to assess SAM's zero-shot performance in segmenting satellite imagery and producing crop-type maps. Although direct crop-type mapping is challenging using SAM in zero-shot setting, experiments reveal SAM's potential for swiftly and accurately outlining fields in satellite images, serving as a foundation for subsequent crop classification. This paper attempts to highlight a use-case of state-of-the-art image segmentation models like SAM for crop-type mapping and related specific needs of the agriculture industry, offering a potential avenue for automatic, efficient, and cost-effective data products for precision agriculture practices.