# FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework

arXiv: https://arxiv.org/abs/2408.06190 · HTML: https://mlsanity.com/paper/2408.06190/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 12 Aug 2024.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [HDR-NeRF: High Dynamic Range Neural Radiance Fields](https://mlsanity.com/paper/2111.14451/) (2021)
- [RobustNeRF: Ignoring Distractors with Robust Losses](https://mlsanity.com/paper/2302.00833/) (2023)
- [Lighting up NeRF via Unsupervised Decomposition and Enhancement](https://mlsanity.com/paper/2307.10664/) (2023)
- [PATHoBot: A Robot for Glasshouse Crop Phenotyping and Intervention](https://mlsanity.com/paper/2010.16272/) (2020)
- [PAg-NeRF: Towards fast and efficient end-to-end panoptic 3D representations for agricultural robotics](https://mlsanity.com/paper/2309.05339/) (2023)
- [Exploring Multi-modal Neural Scene Representations With Applications on Thermal Imaging](https://mlsanity.com/paper/2403.11865/) (2024)
- [Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion](https://mlsanity.com/paper/1804.00307/) (2018)
- [A Comparative Study of Fruit Detection and Counting Methods for Yield Mapping in Apple Orchards](https://mlsanity.com/paper/1810.09499/) (2018)
- [Monocular Camera Based Fruit Counting and Mapping with Semantic Data Association](https://mlsanity.com/paper/1811.01417/) (2018)
- [Apple Counting using Convolutional Neural Networks](https://mlsanity.com/paper/2208.11566/) (2022)

## Built on by (most cited first, 21 of 21)

- [Wheat3DGS: In-field 3D Reconstruction, Instance Segmentation and Phenotyping of Wheat Heads with Gaussian Splatting](https://mlsanity.com/paper/2504.06978/) (2025)
- [A Survey on 3D Reconstruction Techniques in Plant Phenotyping: From Classical Methods to Neural Radiance Fields (NeRF), 3D Gaussian Splatting (3DGS), and Beyond](https://mlsanity.com/paper/2505.00737/) (2025)
- [A survey of datasets for computer vision in agriculture](https://mlsanity.com/paper/2502.16950/) (2025)
- [Multi-Spectral Gaussian Splatting with Neural Color Representation](https://mlsanity.com/paper/2506.03407/) (2025)
- [FruitNeRF++: A Generalized Multi-Fruit Counting Method Utilizing Contrastive Learning and Neural Radiance Fields](https://mlsanity.com/paper/2505.19863/) (2025)
- [Towards Integrating Multi-Spectral Imaging with Gaussian Splatting](https://mlsanity.com/paper/2509.00989/) (2025)
- [InvNeRF-Seg: Fine-Tuning a Pre-Trained NeRF for 3D Object Segmentation](https://mlsanity.com/paper/2504.05751/) (2025)
- [Diffusion-Denoised Hyperspectral Gaussian Splatting](https://mlsanity.com/paper/2505.21890/) (2025)
- [OmniPlantSeg: Species Agnostic 3D Point Cloud Organ Segmentation for High-Resolution Plant Phenotyping Across Modalities](https://mlsanity.com/paper/2509.21038/) (2025)
- [Botany-Bot: Digital Twin Monitoring of Occluded and Underleaf Plant Structures with Gaussian Splats](https://mlsanity.com/paper/2510.17783/) (2025)
- [AgriGS-SLAM: Orchard Mapping Across Seasons via Multi-View Gaussian Splatting SLAM](https://mlsanity.com/paper/2510.26358/) (2025)
- [HS-3D-NeRF: 3D Surface and Hyperspectral Reconstruction From Stationary Hyperspectral Images Using Multi-Channel NeRFs](https://mlsanity.com/paper/2602.16950/) (2026)
- [Vision-based automatic fruit counting with UAV](https://mlsanity.com/paper/2503.13080/) (2025)
- [Cherry Yield Forecast: Harvest Prediction for Individual Sweet Cherry Trees](https://mlsanity.com/paper/2503.20419/) (2025)
- [AppleGrowthVision: A large-scale stereo dataset for phenological analysis, fruit detection, and 3D reconstruction in apple orchards](https://mlsanity.com/paper/2505.14029/) (2025)
- [CountingFruit: Language-Guided 3D Fruit Counting with Semantic Gaussian Splatting](https://mlsanity.com/paper/2506.01109/) (2025)
- [DATR: Diffusion-based 3D Apple Tree Reconstruction Framework with Sparse-View](https://mlsanity.com/paper/2508.19508/) (2025)
- [CropNeRF: A Neural Radiance Field-Based Framework for Crop Counting](https://mlsanity.com/paper/2601.00207/) (2026)
- [From Implicit Ambiguity to Explicit Solidity: Diagnosing Interior Geometric Degradation in Neural Radiance Fields for Dense 3D Scene Understanding](https://mlsanity.com/paper/2601.21421/) (2026)
- [Semantic Radiance Fields as Simulators for Spatial Reasoning in Real-World Scenes](https://mlsanity.com/paper/2608.13095/) (2026)
- [Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs](https://mlsanity.com/paper/2609.07724/) (2026)

## Abstract (arXiv)

We introduce FruitNeRF, a unified novel fruit counting framework that leverages state-of-the-art view synthesis methods to count any fruit type directly in 3D. Our framework takes an unordered set of posed images captured by a monocular camera and segments fruit in each image. To make our system independent of the fruit type, we employ a foundation model that generates binary segmentation masks for any fruit. Utilizing both modalities, RGB and semantic, we train a semantic neural radiance field. Through uniform volume sampling of the implicit Fruit Field, we obtain fruit-only point clouds. By applying cascaded clustering on the extracted point cloud, our approach achieves precise fruit count.The use of neural radiance fields provides significant advantages over conventional methods such as object tracking or optical flow, as the counting itself is lifted into 3D. Our method prevents double counting fruit and avoids counting irrelevant fruit.We evaluate our methodology using both real-world and synthetic datasets. The real-world dataset consists of three apple trees with manually counted ground truths, a benchmark apple dataset with one row and ground truth fruit location, while the synthetic dataset comprises various fruit types including apple, plum, lemon, pear, peach, and mango.Additionally, we assess the performance of fruit counting using the foundation model compared to a U-Net.