# End-to-end deep learning for directly estimating grape yield from ground-based imagery

arXiv: https://arxiv.org/abs/2208.02394 · HTML: https://mlsanity.com/paper/2208.02394/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 53 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [A Comprehensive Survey of Deep Learning for Image Captioning](https://mlsanity.com/paper/1810.04020/) (2018)
- [Visualization of Convolutional Neural Networks for Monocular Depth Estimation](https://mlsanity.com/paper/1904.03380/) (2019)
- [Grape detection, segmentation and tracking using deep neural networks and three-dimensional association](https://mlsanity.com/paper/1907.11819/) (2019)
- [Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards](https://mlsanity.com/paper/1610.08120/) (2016)
- [A Comparative Study of Fruit Detection and Counting Methods for Yield Mapping in Apple Orchards](https://mlsanity.com/paper/1810.09499/) (2018)
- [Enlisting 3D Crop Models and GANs for More Data Efficient and Generalizable Fruit Detection](https://mlsanity.com/paper/2108.13344/) (2021)
- [In-field high throughput grapevine phenotyping with a consumer-grade depth camera](https://mlsanity.com/paper/2104.06945/) (2021)

## Built on by (most cited first, 3 of 3)

- [Benchmarking Edge Computing Devices for Grape Bunches and Trunks Detection using Accelerated Object Detection Single Shot MultiBox Deep Learning Models](https://mlsanity.com/paper/2211.11647/) (2022)
- [Comparing a composite model versus chained models to locate a nearest visual object](https://mlsanity.com/paper/2306.01551/) (2023)
- [Segment Anything for comprehensive analysis of grapevine cluster architecture and berry properties](https://mlsanity.com/paper/2403.12935/) (2024)

## Abstract (arXiv)

Yield estimation is a powerful tool in vineyard management, as it allows growers to fine-tune practices to optimize yield and quality. However, yield estimation is currently performed using manual sampling, which is time-consuming and imprecise. This study demonstrates the application of proximal imaging combined with deep learning for yield estimation in vineyards. Continuous data collection using a vehicle-mounted sensing kit combined with collection of ground truth yield data at harvest using a commercial yield monitor allowed for the generation of a large dataset of 23,581 yield points and 107,933 images. Moreover, this study was conducted in a mechanically managed commercial vineyard, representing a challenging environment for image analysis but a common set of conditions in the California Central Valley. Three model architectures were tested: object detection, CNN regression, and transformer models. The object detection model was trained on hand-labeled images to localize grape bunches, and either bunch count or pixel area was summed to correlate with grape yield. Conversely, regression models were trained end-to-end to predict grape yield from image data without the need for hand labeling. Results demonstrated that both a transformer as well as the object detection model with pixel area processing performed comparably, with a mean absolute percent error of 18% and 18.5%, respectively on a representative holdout dataset. Saliency mapping was used to demonstrate the attention of the CNN model was localized near the predicted location of grape bunches, as well as on the top of the grapevine canopy. Overall, the study showed the applicability of proximal imaging and deep learning for prediction of grapevine yield on a large scale. Additionally, the end-to-end modeling approach was able to perform comparably to the object detection approach while eliminating the need for hand-labeling.