# PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain

arXiv: https://arxiv.org/abs/2306.04557 · HTML: https://mlsanity.com/paper/2306.04557/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 74 of its 139 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 30 of 74)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)

## Built on by (most cited first, 30 of 31)

- [Open-World Semantic Segmentation Including Class Similarity](https://mlsanity.com/paper/2403.07532/) (2024)
- [A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation](https://mlsanity.com/paper/2405.19035/) (2024)
- [BonnBeetClouds3D: A Dataset Towards Point Cloud-based Organ-level Phenotyping of Sugar Beet Plants under Field Conditions](https://mlsanity.com/paper/2312.14706/) (2023)
- [A survey of datasets for computer vision in agriculture](https://mlsanity.com/paper/2502.16950/) (2025)
- [Few-Shot Adaptation of Grounding DINO for Agricultural Domain](https://mlsanity.com/paper/2504.07252/) (2025)
- [LeafNet: A Large-Scale Dataset and Comprehensive Benchmark for Foundational Vision-Language Understanding of Plant Diseases](https://mlsanity.com/paper/2602.13662/) (2026)
- [Zoom in on the Plant: Fine-grained Analysis of Leaf, Stem and Vein Instances](https://mlsanity.com/paper/2312.08805/) (2023)
- [Active learning for efficient annotation in precision agriculture: a use-case on crop-weed semantic segmentation](https://mlsanity.com/paper/2404.02580/) (2024)
- [WeedsGalore: A Multispectral and Multitemporal UAV-based Dataset for Crop and Weed Segmentation in Agricultural Maize Fields](https://mlsanity.com/paper/2502.13103/) (2025)
- [Diffusion Model as a Generalist Segmentation Learner](https://mlsanity.com/paper/2604.24575/) (2026)
- [A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition](https://mlsanity.com/paper/2309.13578/) (2023)
- [A Dataset and Benchmark for Shape Completion of Fruits for Agricultural Robotics](https://mlsanity.com/paper/2407.13304/) (2024)
- [Mitigating Domain Drift in Multi Species Segmentation with DINOv2: A Cross-Domain Evaluation in Herbicide Research Trials](https://mlsanity.com/paper/2508.07514/) (2025)
- [Zero-shot Hierarchical Plant Segmentation via Foundation Segmentation Models and Text-to-image Attention](https://mlsanity.com/paper/2509.09116/) (2025)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [Global Rice Multi-Class Segmentation Dataset (RiceSEG): A Comprehensive and Diverse High-Resolution RGB-Annotated Images for the Development and Benchmarking of Rice Segmentation Algorithms](https://mlsanity.com/paper/2504.02880/) (2025)
- [Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and Leaves](https://mlsanity.com/paper/2310.06582/) (2023)
- [PASTA: Vision Transformer Patch Aggregation for Weakly Supervised Target and Anomaly Segmentation](https://mlsanity.com/paper/2604.09701/) (2026)
- [AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding](https://mlsanity.com/paper/2605.22034/) (2026)
- [Exploiting Boundary Loss for the Hierarchical Panoptic Segmentation of Plants and Leaves](https://mlsanity.com/paper/2501.00527/) (2024)
- [3D Hierarchical Panoptic Segmentation in Real Orchard Environments Across Different Sensors](https://mlsanity.com/paper/2503.13188/) (2025)
- [SemanticSugarBeets: A Multi-Task Framework and Dataset for Inspecting Harvest and Storage Characteristics of Sugar Beets](https://mlsanity.com/paper/2504.16684/) (2025)
- [Lightweight Multispectral Crop-Weed Segmentation for Precision Agriculture](https://mlsanity.com/paper/2505.07444/) (2025)
- [Unlocking Zero-Shot Plant Segmentation with Pl@ntNet Intelligence](https://mlsanity.com/paper/2510.12579/) (2025)
- [DAS-SK: An Adaptive Model Integrating Dual Atrous Separable and Selective Kernel CNN for Agriculture Semantic Segmentation](https://mlsanity.com/paper/2602.08168/) (2026)
- [LeafInst - Unified Instance Segmentation Network for Fine-Grained Forestry Leaf Phenotype Analysis: A New UAV based Benchmark](https://mlsanity.com/paper/2603.03616/) (2026)
- [AgroOmni: A Large-Scale Multi-view Agricultural Dataset for Cross-Scale Multimodal Reasoning](https://mlsanity.com/paper/2603.14342/) (2026)
- [Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment](https://mlsanity.com/paper/2603.23034/) (2026)
- [ReLeaf: Benchmarking Leaf Segmentation across Domains and Species](https://mlsanity.com/paper/2605.03784/) (2026)
- [Self-supervised training for high-resolution close-range multispectral remote sensing imagery](https://mlsanity.com/paper/2607.11366/) (2026)

## Abstract (arXiv)

The production of food, feed, fiber, and fuel is a key task of agriculture, which has to cope with many challenges in the upcoming decades, e.g., a higher demand, climate change, lack of workers, and the availability of arable land. Vision systems can support making better and more sustainable field management decisions, but also support the breeding of new crop varieties by allowing temporally dense and reproducible measurements. Recently, agricultural robotics got an increasing interest in the vision and robotics communities since it is a promising avenue for coping with the aforementioned lack of workers and enabling more sustainable production. While large datasets and benchmarks in other domains are readily available and enable significant progress, agricultural datasets and benchmarks are comparably rare. We present an annotated dataset and benchmarks for the semantic interpretation of real agricultural fields. Our dataset recorded with a UAV provides high-quality, pixel-wise annotations of crops and weeds, but also crop leaf instances at the same time. Furthermore, we provide benchmarks for various tasks on a hidden test set comprised of different fields: known fields covered by the training data and a completely unseen field. Our dataset, benchmarks, and code are available at urlhttps://www.phenobench.org.