# Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain

arXiv: https://arxiv.org/abs/2210.07879 · HTML: https://mlsanity.com/paper/2210.07879/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs](https://mlsanity.com/paper/1709.06764/) (2017)
- [Recurrent Instance Segmentation](https://mlsanity.com/paper/1511.08250/) (2015)
- [Fully Convolutional Networks with Sequential Information for Robust Crop and Weed Detection in Precision Farming](https://mlsanity.com/paper/1806.03412/) (2018)
- [GrowliFlower: An image time series dataset for GROWth analysis of cauLIFLOWER](https://mlsanity.com/paper/2204.00294/) (2022)
- [Instance Segmentation of Biological Images Using Harmonic Embeddings](https://mlsanity.com/paper/1904.05257/) (2019)
- [A Pyramid CNN for Dense-Leaves Segmentation](https://mlsanity.com/paper/1804.01646/) (2018)

## Built on by (most cited first, 17 of 17)

- [PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain](https://mlsanity.com/paper/2306.04557/) (2023)
- [Open-World Semantic Segmentation Including Class Similarity](https://mlsanity.com/paper/2403.07532/) (2024)
- [Adapting the Segment Anything Model During Usage in Novel Situations](https://mlsanity.com/paper/2404.08421/) (2024)
- [BonnBeetClouds3D: A Dataset Towards Point Cloud-based Organ-level Phenotyping of Sugar Beet Plants under Field Conditions](https://mlsanity.com/paper/2312.14706/) (2023)
- [Zoom in on the Plant: Fine-grained Analysis of Leaf, Stem and Vein Instances](https://mlsanity.com/paper/2312.08805/) (2023)
- [A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition](https://mlsanity.com/paper/2309.13578/) (2023)
- [A Dataset and Benchmark for Shape Completion of Fruits for Agricultural Robotics](https://mlsanity.com/paper/2407.13304/) (2024)
- [Zero-shot Hierarchical Plant Segmentation via Foundation Segmentation Models and Text-to-image Attention](https://mlsanity.com/paper/2509.09116/) (2025)
- [Towards Dynamic and Small Objects Refinement for Unsupervised Domain Adaptative Nighttime Semantic Segmentation](https://mlsanity.com/paper/2310.04747/) (2023)
- [Unsupervised Pre-Training for 3D Leaf Instance Segmentation](https://mlsanity.com/paper/2401.08720/) (2024)
- [Open-World Panoptic Segmentation](https://mlsanity.com/paper/2412.12740/) (2024)
- [Joint 3D Point Cloud Segmentation using Real-Sim Loop: From Panels to Trees and Branches](https://mlsanity.com/paper/2503.05630/) (2025)
- [Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and Leaves](https://mlsanity.com/paper/2310.06582/) (2023)
- [Exploiting Boundary Loss for the Hierarchical Panoptic Segmentation of Plants and Leaves](https://mlsanity.com/paper/2501.00527/) (2024)
- [3D Hierarchical Panoptic Segmentation in Real Orchard Environments Across Different Sensors](https://mlsanity.com/paper/2503.13188/) (2025)
- [ReLeaf: Benchmarking Leaf Segmentation across Domains and Species](https://mlsanity.com/paper/2605.03784/) (2026)
- [Text-conditioned Segmentation for Tomato Phenotyping via Procedural Synthetic Data](https://mlsanity.com/paper/2607.18576/) (2026)

## Abstract (arXiv)

Plant phenotyping is a central task in agriculture, as it describes plants' growth stage, development, and other relevant quantities. Robots can help automate this process by accurately estimating plant traits such as the number of leaves, leaf area, and the plant size. In this paper, we address the problem of joint semantic, plant instance, and leaf instance segmentation of crop fields from RGB data. We propose a single convolutional neural network that addresses the three tasks simultaneously, exploiting their underlying hierarchical structure. We introduce task-specific skip connections, which our experimental evaluation proves to be more beneficial than the usual schemes. We also propose a novel automatic post-processing, which explicitly addresses the problem of spatially close instances, common in the agricultural domain because of overlapping leaves. Our architecture simultaneously tackles these problems jointly in the agricultural context. Previous works either focus on plant or leaf segmentation, or do not optimise for semantic segmentation. Results show that our system has superior performance compared to state-of-the-art approaches, while having a reduced number of parameters and is operating at camera frame rate.