# Explicitly incorporating spatial information to recurrent networks for agriculture

arXiv: https://arxiv.org/abs/2206.13406 · HTML: https://mlsanity.com/paper/2206.13406/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Delving Deeper into Convolutional Networks for Learning Video Representations](https://mlsanity.com/paper/1511.06432/) (2015)
- [Self-Supervised Model Adaptation for Multimodal Semantic Segmentation](https://mlsanity.com/paper/1808.03833/) (2018)
- [DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks](https://mlsanity.com/paper/1703.03098/) (2017)
- [PATHoBot: A Robot for Glasshouse Crop Phenotyping and Intervention](https://mlsanity.com/paper/2010.16272/) (2020)
- [Autonomous Sweet Pepper Harvesting for Protected Cropping Systems](https://mlsanity.com/paper/1706.02023/) (2017)
- [Recurrent Convolutional Fusion for RGB-D Object Recognition](https://mlsanity.com/paper/1806.01673/) (2018)
- [Virtual Temporal Samples for Recurrent Neural Networks: applied to semantic segmentation in agriculture](https://mlsanity.com/paper/2106.10118/) (2021)

## Built on by (most cited first, 4 of 4)

- [PAg-NeRF: Towards fast and efficient end-to-end panoptic 3D representations for agricultural robotics](https://mlsanity.com/paper/2309.05339/) (2023)
- [Panoptic One-Click Segmentation: Applied to Agricultural Data](https://mlsanity.com/paper/2303.08689/) (2023)
- [Spatial Information Integration in Small Language Models for Document Layout Generation and Classification](https://mlsanity.com/paper/2501.05497/) (2025)
- [DropClick: Semi-Automated One-Click Segmentation for Agricultural Robotic Data](https://mlsanity.com/paper/2609.03680/) (2026)

## Abstract (arXiv)

In agriculture, the majority of vision systems perform still image classification. Yet, recent work has highlighted the potential of spatial and temporal cues as a rich source of information to improve the classification performance. In this paper, we propose novel approaches to explicitly capture both spatial and temporal information to improve the classification of deep convolutional neural networks. We leverage available RGB-D images and robot odometry to perform inter-frame feature map spatial registration. This information is then fused within recurrent deep learnt models, to improve their accuracy and robustness. We demonstrate that this can considerably improve the classification performance with our best performing spatial-temporal model (ST-Atte) achieving absolute performance improvements for intersection-over-union (IoU[%]) of 4.7 for crop-weed segmentation and 2.6 for fruit (sweet pepper) segmentation. Furthermore, we show that these approaches are robust to variable framerates and odometry errors, which are frequently observed in real-world applications.