# Grape detection, segmentation and tracking using deep neural networks and three-dimensional association

arXiv: https://arxiv.org/abs/1907.11819 · HTML: https://mlsanity.com/paper/1907.11819/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2019.

## Built on by (most cited first, 21 of 21)

- [Label-Efficient Learning in Agriculture: A Comprehensive Review](https://mlsanity.com/paper/2305.14691/) (2023)
- [Enlisting 3D Crop Models and GANs for More Data Efficient and Generalizable Fruit Detection](https://mlsanity.com/paper/2108.13344/) (2021)
- [Deep-Learning-based Counting Methods, Datasets, and Applications in Agriculture -- A Review](https://mlsanity.com/paper/2303.02632/) (2023)
- [Weakly and Semi-Supervised Detection, Segmentation and Tracking of Table Grapes with Limited and Noisy Data](https://mlsanity.com/paper/2208.13001/) (2022)
- [Few-Shot Adaptation of Grounding DINO for Agricultural Domain](https://mlsanity.com/paper/2504.07252/) (2025)
- [DT/MARS-CycleGAN: Improved Object Detection for MARS Phenotyping Robot](https://mlsanity.com/paper/2310.12787/) (2023)
- [End-to-end deep learning for directly estimating grape yield from ground-based imagery](https://mlsanity.com/paper/2208.02394/) (2022)
- [Grapevine Winter Pruning Automation: On Potential Pruning Points Detection through 2D Plant Modeling using Grapevine Segmentation](https://mlsanity.com/paper/2106.04208/) (2021)
- [A methodology for detection and localization of fruits in apples orchards from aerial images](https://mlsanity.com/paper/2110.12331/) (2021)
- [A differentiable Gaussian Prototype Layer for explainable Segmentation](https://mlsanity.com/paper/2306.14361/) (2023)
- [Detecting Olives with Synthetic or Real Data? Olive the Above](https://mlsanity.com/paper/2308.08271/) (2023)
- [How to Train an Accurate and Efficient Object Detection Model on Any Dataset](https://mlsanity.com/paper/2211.17170/) (2022)
- [Surgical fine-tuning for Grape Bunch Segmentation under Visual Domain Shifts](https://mlsanity.com/paper/2307.00837/) (2023)
- [SPROUT: A Scalable Diffusion Foundation Model for Multi-Crop Plant Phenotyping](https://mlsanity.com/paper/2603.27519/) (2026)
- [Attention-based fusion of semantic boundary and non-boundary information to improve semantic segmentation](https://mlsanity.com/paper/2108.02840/) (2021)
- [Towards Precise Pruning Points Detection using Semantic-Instance-Aware Plant Models for Grapevine Winter Pruning Automation](https://mlsanity.com/paper/2109.07247/) (2021)
- [LDD: A Dataset for Grape Diseases Object Detection and Instance Segmentation](https://mlsanity.com/paper/2206.10192/) (2022)
- [3D Pose Estimation of Tomato Peduncle Nodes using Deep Keypoint Detection and Point Cloud](https://mlsanity.com/paper/2311.04699/) (2023)
- [Phenology-based learning framework for yield estimation and harvest forecasting of raspberry fruits](https://mlsanity.com/paper/2411.00967/) (2024)
- [Advancing Image-Based Grapevine Variety Classification with a New Benchmark and Evaluation of Masked Autoencoders](https://mlsanity.com/paper/2506.13335/) (2025)
- [ViViD-5K: Vineyard vision dataset for field-based berry detection and segmentation and grape cluster closure estimation](https://mlsanity.com/paper/2605.24353/) (2026)

## Abstract (arXiv)

Agricultural applications such as yield prediction, precision agriculture and automated harvesting need systems able to infer the crop state from low-cost sensing devices. Proximal sensing using affordable cameras combined with computer vision has seen a promising alternative, strengthened after the advent of convolutional neural networks (CNNs) as an alternative for challenging pattern recognition problems in natural images. Considering fruit growing monitoring and automation, a fundamental problem is the detection, segmentation and counting of individual fruits in orchards. Here we show that for wine grapes, a crop presenting large variability in shape, color, size and compactness, grape clusters can be successfully detected, segmented and tracked using state-of-the-art CNNs. In a test set containing 408 grape clusters from images taken on a trellis-system based vineyard, we have reached an F 1 -score up to 0.91 for instance segmentation, a fine separation of each cluster from other structures in the image that allows a more accurate assessment of fruit size and shape. We have also shown as clusters can be identified and tracked along video sequences recording orchard rows. We also present a public dataset containing grape clusters properly annotated in 300 images and a novel annotation methodology for segmentation of complex objects in natural images. The presented pipeline for annotation, training, evaluation and tracking of agricultural patterns in images can be replicated for different crops and production systems. It can be employed in the development of sensing components for several agricultural and environmental applications.