# Push Past Green: Learning to Look Behind Plant Foliage by Moving It

arXiv: https://arxiv.org/abs/2307.03175 · HTML: https://mlsanity.com/paper/2307.03175/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 57 references that are in the map.
- First posted on arXiv 6 Jul 2023.

## Builds on (most cited first, 30 of 36)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Unsupervised Learning for Physical Interaction through Video Prediction](https://mlsanity.com/paper/1605.07157/) (2016)
- [Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control](https://mlsanity.com/paper/1812.00568/) (2018)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories](https://mlsanity.com/paper/2204.04153/) (2022)
- [Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours](https://mlsanity.com/paper/1509.06825/) (2015)
- [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/2011.07215/) (2020)
- [Visual Imitation Made Easy](https://mlsanity.com/paper/2008.04899/) (2020)
- [Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations](https://mlsanity.com/paper/1912.04344/) (2019)
- [Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation](https://mlsanity.com/paper/1703.02018/) (2017)
- [FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding](https://mlsanity.com/paper/2105.03655/) (2021)
- [RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks](https://mlsanity.com/paper/2205.02909/) (2022)
- [Propagation Networks for Model-Based Control Under Partial Observation](https://mlsanity.com/paper/1809.11169/) (2018)
- [FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy](https://mlsanity.com/paper/2111.05623/) (2021)
- [DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools](https://mlsanity.com/paper/2203.17275/) (2022)
- [Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objects](https://mlsanity.com/paper/2203.00663/) (2022)
- [Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation](https://mlsanity.com/paper/2206.02881/) (2022)
- [A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy](https://mlsanity.com/paper/2011.02398/) (2020)
- [Learned Visual Navigation for Under-Canopy Agricultural Robots](https://mlsanity.com/paper/2107.02792/) (2021)
- [DextAIRity: Deformable Manipulation Can be a Breeze](https://mlsanity.com/paper/2203.01197/) (2022)
- [A Robust Illumination-Invariant Camera System for Agricultural Applications](https://mlsanity.com/paper/2101.02190/) (2021)
- [Bumblebee: A Path Towards Fully Autonomous Robotic Vine Pruning](https://mlsanity.com/paper/2112.00291/) (2021)
- [Learning Robotic Manipulation of Granular Media](https://mlsanity.com/paper/1709.02833/) (2017)
- [TrimBot2020: an outdoor robot for automatic gardening](https://mlsanity.com/paper/1804.01792/) (2018)
- [Visual Perception and Modelling in Unstructured Orchard for Apple Harvesting Robots](https://mlsanity.com/paper/1912.12555/) (2019)
- [Multi-Sensor Fusion based Robust Row Following for Compact Agricultural Robots](https://mlsanity.com/paper/2106.15029/) (2021)

## Built on by (most cited first, 3 of 3)

- [Find the Fruit: Zero-Shot Sim2Real RL for Occlusion-Aware Plant Manipulation](https://mlsanity.com/paper/2505.16547/) (2025)
- [Deformable Cluster Manipulation via Whole-Arm Policy Learning](https://mlsanity.com/paper/2507.17085/) (2025)
- [Reinforcement Learning for the Full Strawberry Harvesting Process: Obstacle Separation, Detachment, and Placement](https://mlsanity.com/paper/2607.14708/) (2026)

## Abstract (arXiv)

Autonomous agriculture applications (e.g., inspection, phenotyping, plucking fruits) require manipulating the plant foliage to look behind the leaves and the branches. Partial visibility, extreme clutter, thin structures, and unknown geometry and dynamics for plants make such manipulation challenging. We tackle these challenges through data-driven methods. We use self-supervision to train SRPNet, a neural network that predicts what space is revealed on execution of a candidate action on a given plant. We use SRPNet with the cross-entropy method to predict actions that are effective at revealing space beneath plant foliage. Furthermore, as SRPNet does not just predict how much space is revealed but also where it is revealed, we can execute a sequence of actions that incrementally reveal more and more space beneath the plant foliage. We experiment with a synthetic (vines) and a real plant (Dracaena) on a physical test-bed across 5 settings including 2 settings that test generalization to novel plant configurations. Our experiments reveal the effectiveness of our overall method, PPG, over a competitive hand-crafted exploration method, and the effectiveness of SRPNet over a hand-crafted dynamics model and relevant ablations.