# Learned Visual Navigation for Under-Canopy Agricultural Robots

arXiv: https://arxiv.org/abs/2107.02792 · HTML: https://mlsanity.com/paper/2107.02792/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 75 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Towards End-to-End Lane Detection: an Instance Segmentation Approach](https://mlsanity.com/paper/1802.05591/) (2018)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [Driving Policy Transfer via Modularity and Abstraction](https://mlsanity.com/paper/1804.09364/) (2018)
- [Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis](https://mlsanity.com/paper/2001.01306/) (2020)
- [Combining Optimal Control and Learning for Visual Navigation in Novel Environments](https://mlsanity.com/paper/1903.02531/) (2019)
- [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017)
- [Robust Lane Detection from Continuous Driving Scenes Using Deep Neural Networks](https://mlsanity.com/paper/1903.02193/) (2019)
- [Scaling Local Control to Large-Scale Topological Navigation](https://mlsanity.com/paper/1909.12329/) (2019)
- [Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents](https://mlsanity.com/paper/2009.03231/) (2020)
- [Visual Memory for Robust Path Following](https://mlsanity.com/paper/1812.00940/) (2018)
- [Learning to Move with Affordance Maps](https://mlsanity.com/paper/2001.02364/) (2020)
- [The Rosario Dataset: Multisensor Data for Localization and Mapping in Agricultural Environments](https://mlsanity.com/paper/1809.06413/) (2018)
- [Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues](https://mlsanity.com/paper/1708.03798/) (2017)
- [Local Motion Planner for Autonomous Navigation in Vineyards with a RGB-D Camera-Based Algorithm and Deep Learning Synergy](https://mlsanity.com/paper/2005.12815/) (2020)
- [Multi-Sensor Fusion based Robust Row Following for Compact Agricultural Robots](https://mlsanity.com/paper/2106.15029/) (2021)
- [High Precision Control of Tracked Field Robots in the Presence of Unknown Traction Coefficients](https://mlsanity.com/paper/2103.11294/) (2021)
- [Design and Construction of Unmanned Ground Vehicles for Sub-Canopy Plant Phenotyping](https://mlsanity.com/paper/1903.10608/) (2019)

## Built on by (most cited first, 18 of 18)

- [WayFAST: Navigation with Predictive Traversability in the Field](https://mlsanity.com/paper/2203.12071/) (2022)
- [Proactive Anomaly Detection for Robot Navigation with Multi-Sensor Fusion](https://mlsanity.com/paper/2204.01146/) (2022)
- [Towards agricultural autonomy: crop row detection under varying field conditions using deep learning](https://mlsanity.com/paper/2109.08247/) (2021)
- [Vision-based Vineyard Navigation Solution with Automatic Annotation](https://mlsanity.com/paper/2303.14347/) (2023)
- [Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation](https://mlsanity.com/paper/2411.04112/) (2024)
- [Towards Infield Navigation: leveraging simulated data for crop row detection](https://mlsanity.com/paper/2204.01811/) (2022)
- [Push Past Green: Learning to Look Behind Plant Foliage by Moving It](https://mlsanity.com/paper/2307.03175/) (2023)
- [Lessons from Deploying CropFollow++: Under-Canopy Agricultural Navigation with Keypoints](https://mlsanity.com/paper/2404.17718/) (2024)
- [ZeST: A VLM-based Zero-Shot Traversability Navigation for Unknown Environments](https://mlsanity.com/paper/2508.19131/) (2025)
- [A Deep Learning Driven Algorithmic Pipeline for Autonomous Navigation in Row-Based Crops](https://mlsanity.com/paper/2112.03816/) (2021)
- [RowDetr: End-to-End Crop Row Detection Using Polynomials](https://mlsanity.com/paper/2412.10525/) (2024)
- [End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2509.18608/) (2025)
- [Spannotation: Enhancing Semantic Segmentation for Autonomous Navigation with Efficient Image Annotation](https://mlsanity.com/paper/2402.18084/) (2024)
- [Towards Real-Time Generation of Delay-Compensated Video Feeds for Outdoor Mobile Robot Teleoperation](https://mlsanity.com/paper/2409.09921/) (2024)
- [CA-Cut: Crop-Aligned Cutout for Data Augmentation to Learn More Robust Under-Canopy Navigation](https://mlsanity.com/paper/2507.17727/) (2025)
- [LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields](https://mlsanity.com/paper/2606.31941/) (2026)
- [Autonomous Agricultural Tractor: Integrated Weed Detection and LiDAR Navigation for Precision Paddy Farming](https://mlsanity.com/paper/2608.19004/) (2026)
- [ObstaDiff: Generalizable Diffusion Policy Learning via Obstacle-aware Representations](https://mlsanity.com/paper/2609.10918/) (2026)

## Abstract (arXiv)

We describe a system for visually guided autonomous navigation of under-canopy farm robots. Low-cost under-canopy robots can drive between crop rows under the plant canopy and accomplish tasks that are infeasible for over-the-canopy drones or larger agricultural equipment. However, autonomously navigating them under the canopy presents a number of challenges: unreliable GPS and LiDAR, high cost of sensing, challenging farm terrain, clutter due to leaves and weeds, and large variability in appearance over the season and across crop types. We address these challenges by building a modular system that leverages machine learning for robust and generalizable perception from monocular RGB images from low-cost cameras, and model predictive control for accurate control in challenging terrain. Our system, CropFollow, is able to autonomously drive 485 meters per intervention on average, outperforming a state-of-the-art LiDAR based system (286 meters per intervention) in extensive field testing spanning over 25 km.