# Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs

arXiv: https://arxiv.org/abs/1709.06764 · HTML: https://mlsanity.com/paper/1709.06764/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2017.

## Built on by (most cited first, 30 of 55)

- [Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation](https://mlsanity.com/paper/2103.04705/) (2021)
- [PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain](https://mlsanity.com/paper/2306.04557/) (2023)
- [Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attention](https://mlsanity.com/paper/2303.13014/) (2023)
- [Open-World Semantic Segmentation Including Class Similarity](https://mlsanity.com/paper/2403.07532/) (2024)
- [Hyperbolic Uncertainty Aware Semantic Segmentation](https://mlsanity.com/paper/2203.08881/) (2022)
- [Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain](https://mlsanity.com/paper/2210.07879/) (2022)
- [GrowliFlower: An image time series dataset for GROWth analysis of cauLIFLOWER](https://mlsanity.com/paper/2204.00294/) (2022)
- [WeatherProof: A Paired-Dataset Approach to Semantic Segmentation in Adverse Weather](https://mlsanity.com/paper/2312.09534/) (2023)
- [Augmentation Invariance and Adaptive Sampling in Semantic Segmentation of Agricultural Aerial Images](https://mlsanity.com/paper/2204.07969/) (2022)
- [Semi-Supervised Semantic Segmentation via Marginal Contextual Information](https://mlsanity.com/paper/2308.13900/) (2023)
- [Potato Crop Stress Identification in Aerial Images using Deep Learning-based Object Detection](https://mlsanity.com/paper/2106.07770/) (2021)
- [Semantic Scene Segmentation for Robotics Applications](https://mlsanity.com/paper/2108.11128/) (2021)
- [On Efficient Real-Time Semantic Segmentation: A Survey](https://mlsanity.com/paper/2206.08605/) (2022)
- [Standardizing and Centralizing Datasets to Enable Efficient Training of Agricultural Deep Learning Models](https://mlsanity.com/paper/2208.02707/) (2022)
- [Semi-supervised dry herbage mass estimation using automatic data and synthetic images](https://mlsanity.com/paper/2110.13719/) (2021)
- [Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation](https://mlsanity.com/paper/2208.06820/) (2022)
- [BonnBeetClouds3D: A Dataset Towards Point Cloud-based Organ-level Phenotyping of Sugar Beet Plants under Field Conditions](https://mlsanity.com/paper/2312.14706/) (2023)
- [A survey of datasets for computer vision in agriculture](https://mlsanity.com/paper/2502.16950/) (2025)
- [Lightweight Distributed Gaussian Process Regression for Online Machine Learning](https://mlsanity.com/paper/2105.04738/) (2021)
- [WeedsGalore: A Multispectral and Multitemporal UAV-based Dataset for Crop and Weed Segmentation in Agricultural Maize Fields](https://mlsanity.com/paper/2502.13103/) (2025)
- [Towards practical object detection for weed spraying in precision agriculture](https://mlsanity.com/paper/2109.11048/) (2021)
- [Uncertainty in Real-Time Semantic Segmentation on Embedded Systems](https://mlsanity.com/paper/2301.01201/) (2022)
- [iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species](https://mlsanity.com/paper/2503.20068/) (2025)
- [Grapevine Winter Pruning Automation: On Potential Pruning Points Detection through 2D Plant Modeling using Grapevine Segmentation](https://mlsanity.com/paper/2106.04208/) (2021)
- [Labeling Where Adapting Fails: Cross-Domain Semantic Segmentation with Point Supervision via Active Selection](https://mlsanity.com/paper/2206.00181/) (2022)
- [Relative Difficulty Distillation for Semantic Segmentation](https://mlsanity.com/paper/2407.03719/) (2024)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [Vision Foundation Models in Agriculture: Toward Domain-Specific Adaptation for Weed Herbicide Trials Assessment](https://mlsanity.com/paper/2511.04288/) (2025)
- [Using depth information and colour space variations for improving outdoor robustness for instance segmentation of cabbage](https://mlsanity.com/paper/2103.16923/) (2021)
- [Deep-CNN based Robotic Multi-Class Under-Canopy Weed Control in Precision Farming](https://mlsanity.com/paper/2112.13986/) (2021)

## Abstract (arXiv)

Precision farming robots, which target to reduce the amount of herbicides that need to be brought out in the fields, must have the ability to identify crops and weeds in real time to trigger weeding actions. In this paper, we address the problem of CNN-based semantic segmentation of crop fields separating sugar beet plants, weeds, and background solely based on RGB data. We propose a CNN that exploits existing vegetation indexes and provides a classification in real time. Furthermore, it can be effectively re-trained to so far unseen fields with a comparably small amount of training data. We implemented and thoroughly evaluated our system on a real agricultural robot operating in different fields in Germany and Switzerland. The results show that our system generalizes well, can operate at around 20Hz, and is suitable for online operation in the fields.