# Optical flow-based branch segmentation for complex orchard environments

arXiv: https://arxiv.org/abs/2202.13050 · HTML: https://mlsanity.com/paper/2202.13050/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 26 Feb 2022.

## Builds on (most cited first, 11 of 11)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [Video Object Segmentation and Tracking: A Survey](https://mlsanity.com/paper/1904.09172/) (2019)
- [Background Subtraction in Real Applications: Challenges, Current Models and Future Directions](https://mlsanity.com/paper/1901.03577/) (2019)
- [Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation](https://mlsanity.com/paper/1811.05255/) (2018)
- [Semantic Segmentation for Partially Occluded Apple Trees Based on Deep Learning](https://mlsanity.com/paper/2010.06879/) (2020)
- [Precision fruit tree pruning using a learned hybrid vision/interaction controller](https://mlsanity.com/paper/2109.13162/) (2021)
- [A robotic vision system to measure tree traits](https://mlsanity.com/paper/1707.05368/) (2017)

## Built on by (most cited first, 4 of 4)

- [Joint 3D Point Cloud Segmentation using Real-Sim Loop: From Panels to Trees and Branches](https://mlsanity.com/paper/2503.05630/) (2025)
- [Learning to Prune Branches in Modern Tree-Fruit Orchards](https://mlsanity.com/paper/2507.23015/) (2025)
- [Towards scalable organ level 3D plant segmentation: Bridging the data algorithm computing gap](https://mlsanity.com/paper/2509.06329/) (2025)
- [Visuomotor Robotic Pruning in Planar Orchards Using Hybrid Reinforcement Learning](https://mlsanity.com/paper/2609.24906/) (2026)

## Abstract (arXiv)

Machine vision is a critical subsystem for enabling robots to be able to perform a variety of tasks in orchard environments. However, orchards are highly visually complex environments, and computer vision algorithms operating in them must be able to contend with variable lighting conditions and background noise. Past work on enabling deep learning algorithms to operate in these environments has typically required large amounts of hand-labeled data to train a deep neural network or physically controlling the conditions under which the environment is perceived. In this paper, we train a neural network system in simulation only using simulated RGB data and optical flow. This resulting neural network is able to perform foreground segmentation of branches in a busy orchard environment without additional real-world training or using any special setup or equipment beyond a standard camera. Our results show that our system is highly accurate and, when compared to a network using manually labeled RGBD data, achieves significantly more consistent and robust performance across environments that differ from the training set.