# Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion

arXiv: https://arxiv.org/abs/1804.00307 · HTML: https://mlsanity.com/paper/1804.00307/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2018.

## Built on by (most cited first, 13 of 13)

- [FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework](https://mlsanity.com/paper/2408.06190/) (2024)
- [Scene-Adaptive Attention Network for Crowd Counting](https://mlsanity.com/paper/2112.15509/) (2021)
- [CitDet: A Benchmark Dataset for Citrus Fruit Detection](https://mlsanity.com/paper/2309.05645/) (2023)
- [FruitNeRF++: A Generalized Multi-Fruit Counting Method Utilizing Contrastive Learning and Neural Radiance Fields](https://mlsanity.com/paper/2505.19863/) (2025)
- [High precision control and deep learning-based corn stand counting algorithms for agricultural robot](https://mlsanity.com/paper/2103.11276/) (2021)
- [Cross-Region Building Counting in Satellite Imagery using Counting Consistency](https://mlsanity.com/paper/2110.13558/) (2021)
- [Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2209.10003/) (2022)
- [Autonomous Strawberry Picking Robotic System (Robofruit)](https://mlsanity.com/paper/2301.03947/) (2023)
- [Why Change Your Controller When You Can Change Your Planner: Drag-Aware Trajectory Generation for Quadrotor Systems](https://mlsanity.com/paper/2401.04960/) (2024)
- [Few-Shot Fruit Segmentation via Transfer Learning](https://mlsanity.com/paper/2405.02556/) (2024)
- [Advanced techniques and applications of LiDAR Place Recognition in Agricultural Environments: A Comprehensive Survey](https://mlsanity.com/paper/2601.22198/) (2026)
- [CountingFruit: Language-Guided 3D Fruit Counting with Semantic Gaussian Splatting](https://mlsanity.com/paper/2506.01109/) (2025)
- [DATR: Diffusion-based 3D Apple Tree Reconstruction Framework with Sparse-View](https://mlsanity.com/paper/2508.19508/) (2025)

## Abstract (arXiv)

We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.