# Enlisting 3D Crop Models and GANs for More Data Efficient and Generalizable Fruit Detection

arXiv: https://arxiv.org/abs/2108.13344 · HTML: https://mlsanity.com/paper/2108.13344/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 15 of 15)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [Unsupervised Cross-Domain Image Generation](https://mlsanity.com/paper/1611.02200/) (2016)
- [CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency](https://mlsanity.com/paper/2001.03182/) (2020)
- [Deep Fruit Detection in Orchards](https://mlsanity.com/paper/1610.03677/) (2016)
- [Grape detection, segmentation and tracking using deep neural networks and three-dimensional association](https://mlsanity.com/paper/1907.11819/) (2019)
- [Temporal Prediction and Evaluation of Brassica Growth in the Field using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2105.07789/) (2021)
- [Counting of Grapevine Berries in Images via Semantic Segmentation using Convolutional Neural Networks](https://mlsanity.com/paper/2004.14010/) (2020)
- [A Robust Illumination-Invariant Camera System for Agricultural Applications](https://mlsanity.com/paper/2101.02190/) (2021)
- [Behind the leaves -- Estimation of occluded grapevine berries with conditional generative adversarial networks](https://mlsanity.com/paper/2105.10325/) (2021)

## Built on by (most cited first, 9 of 9)

- [Standardizing and Centralizing Datasets to Enable Efficient Training of Agricultural Deep Learning Models](https://mlsanity.com/paper/2208.02707/) (2022)
- [Generating Diverse Agricultural Data for Vision-Based Farming Applications](https://mlsanity.com/paper/2403.18351/) (2024)
- [Learn from Foundation Model: Fruit Detection Model without Manual Annotation](https://mlsanity.com/paper/2411.16196/) (2024)
- [Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review](https://mlsanity.com/paper/2204.04707/) (2022)
- [End-to-end deep learning for directly estimating grape yield from ground-based imagery](https://mlsanity.com/paper/2208.02394/) (2022)
- [LAESI: Leaf Area Estimation with Synthetic Imagery](https://mlsanity.com/paper/2404.00593/) (2024)
- [AGILE: A Diffusion-Based Attention-Guided Image and Label Translation for Efficient Cross-Domain Plant Trait Identification](https://mlsanity.com/paper/2503.22019/) (2025)
- [A workflow for segmenting soil and plant X-ray CT images with deep learning in Googles Colaboratory](https://mlsanity.com/paper/2203.09674/) (2022)
- [Text-conditioned Segmentation for Tomato Phenotyping via Procedural Synthetic Data](https://mlsanity.com/paper/2607.18576/) (2026)

## Abstract (arXiv)

Training real-world neural network models to achieve high performance and generalizability typically requires a substantial amount of labeled data, spanning a broad range of variation. This data-labeling process can be both labor and cost intensive. To achieve desirable predictive performance, a trained model is typically applied into a domain where the data distribution is similar to the training dataset. However, for many agricultural machine learning problems, training datasets are collected at a specific location, during a specific period in time of the growing season. Since agricultural systems exhibit substantial variability in terms of crop type, cultivar, management, seasonal growth dynamics, lighting condition, sensor type, etc, a model trained from one dataset often does not generalize well across domains. To enable more data efficient and generalizable neural network models in agriculture, we propose a method that generates photorealistic agricultural images from a synthetic 3D crop model domain into real world crop domains. The method uses a semantically constrained GAN (generative adversarial network) to preserve the fruit position and geometry. We observe that a baseline CycleGAN method generates visually realistic target domain images but does not preserve fruit position information while our method maintains fruit positions well. Image generation results in vineyard grape day and night images show the visual outputs of our network are much better compared to a baseline network. Incremental training experiments in vineyard grape detection tasks show that the images generated from our method can significantly speed the domain adaption process, increase performance for a given number of labeled images (i.e. data efficiency), and decrease labeling requirements.