# Hierarchical Mask2Former: Panoptic Segmentation of Crops, Weeds and Leaves

arXiv: https://arxiv.org/abs/2310.06582 · HTML: https://mlsanity.com/paper/2310.06582/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain](https://mlsanity.com/paper/2306.04557/) (2023)
- [weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming](https://mlsanity.com/paper/1709.03329/) (2017)
- [Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain](https://mlsanity.com/paper/2210.07879/) (2022)
- [Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection](https://mlsanity.com/paper/1612.03019/) (2016)
- [Leaf Counting with Deep Convolutional and Deconvolutional Networks](https://mlsanity.com/paper/1708.07570/) (2017)

## Built on by (most cited first, 1 of 1)

- [A Simple and Generalist Approach for Panoptic Segmentation](https://mlsanity.com/paper/2408.16504/) (2024)

## Abstract (arXiv)

Advancements in machine vision that enable detailed inferences to be made from images have the potential to transform many sectors including agriculture. Precision agriculture, where data analysis enables interventions to be precisely targeted, has many possible applications. Precision spraying, for example, can limit the application of herbicide only to weeds, or limit the application of fertiliser only to undernourished crops, instead of spraying the entire field. The approach promises to maximise yields, whilst minimising resource use and harms to the surrounding environment. To this end, we propose a hierarchical panoptic segmentation method to simultaneously identify indicators of plant growth and locate weeds within an image. We adapt Mask2Former, a state-of-the-art architecture for panoptic segmentation, to predict crop, weed and leaf masks. We achieve a PQdag of 75.99. Additionally, we explore approaches to make the architecture more compact and therefore more suitable for time and compute constrained applications. With our more compact architecture, inference is up to 60% faster and the reduction in PQdag is less than 1%.