# Size Doesn't Matter: Material-State Reinforcement Learning for Excavator Transferable Soil Manipulation

arXiv: https://arxiv.org/abs/2609.12677 · HTML: https://mlsanity.com/paper/2609.12677/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 11 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 33)

- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations](https://mlsanity.com/paper/1912.04344/) (2019)
- [PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics](https://mlsanity.com/paper/2104.03311/) (2021)
- [RSL-RL: A Learning Library for Robotics Research](https://mlsanity.com/paper/2509.10771/) (2025)
- [FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation](https://mlsanity.com/paper/2303.02346/) (2023)
- [Inferring the Material Properties of Granular Media for Robotic Tasks](https://mlsanity.com/paper/2003.08032/) (2020)
- [Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation](https://mlsanity.com/paper/2412.12089/) (2024)
- [Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots](https://mlsanity.com/paper/2203.01389/) (2022)
- [Optimization-Based Framework for Excavation Trajectory Generation](https://mlsanity.com/paper/2010.14038/) (2020)
- [Real-Time Motion Planning of a Hydraulic Excavator using Trajectory Optimization and Model Predictive Control](https://mlsanity.com/paper/2107.02366/) (2021)
- [A multiscale model of terrain dynamics for real-time earthmoving simulation](https://mlsanity.com/paper/2011.00459/) (2020)
- [Dynamic Modeling of Bucket-Soil Interactions Using Koopman-DFL Lifting Linearization for Model Predictive Contouring Control of Autonomous Excavators](https://mlsanity.com/paper/2107.04314/) (2021)
- [ExACT: An End-to-End Autonomous Excavator System Using Action Chunking With Transformers](https://mlsanity.com/paper/2405.05861/) (2024)
- [Granular Gym: High Performance Simulation for Robotic Tasks with Granular Materials](https://mlsanity.com/paper/2306.01369/) (2023)
- [Examining the simulation-to-reality gap of a wheel loader digging in deformable terrain](https://mlsanity.com/paper/2310.05765/) (2023)
- [Large Scale Robotic Material Handling: Learning, Planning, and Control](https://mlsanity.com/paper/2508.09003/) (2025)
- [Towards Learning Boulder Excavation with Hydraulic Excavators](https://mlsanity.com/paper/2509.17683/) (2025)
- [Towards Autonomous Grading In The Real World](https://mlsanity.com/paper/2206.06091/) (2022)
- [Optimal Control of Granular Material](https://mlsanity.com/paper/2302.03231/) (2023)
- [Progressive-Resolution Policy Distillation: Leveraging Coarse-Resolution Simulations for Time-Efficient Fine-Resolution Policy Learning](https://mlsanity.com/paper/2412.07477/) (2024)
- [A Physics-Based Continuum Model for Versatile, Scalable, and Fast Terramechanics Simulation](https://mlsanity.com/paper/2507.05643/) (2025)
- [ExT: Towards Scalable Autonomous Excavation via Large-Scale Multi-Task Pretraining and Fine-Tuning](https://mlsanity.com/paper/2509.14992/) (2025)
- [DDBot: Differentiable Physics-based Digging Robot for Unknown Granular Materials](https://mlsanity.com/paper/2510.17335/) (2025)
- [Differentiable Material Point Method for the Control of Deformable Objects](https://mlsanity.com/paper/2512.13214/) (2025)
- [In Situ Soil Property Estimation for Autonomous Earthmoving Using Physics-Infused Neural Networks](https://mlsanity.com/paper/2309.02575/) (2023)
- [Autonomous Excavation of Challenging Terrain using Oscillatory Primitives and Adaptive Impedance Control](https://mlsanity.com/paper/2409.18273/) (2024)
- [A Convex Formulation of Material Points and Rigid Bodies with GPU-Accelerated Async-Coupling for Interactive Simulation](https://mlsanity.com/paper/2503.05046/) (2025)
- [GeoWarp: An automatically differentiable and GPU-accelerated implicit MPM framework for geomechanics based on NVIDIA Warp](https://mlsanity.com/paper/2507.09435/) (2025)
- [Interactive Shaping of Granular Media Using Reinforcement Learning](https://mlsanity.com/paper/2509.06469/) (2025)
- [Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation](https://mlsanity.com/paper/2510.04168/) (2025)

## Abstract (arXiv)

Earthmoving tasks such as excavation, backfilling, or embankment construction require deliberate repositioning of deformable soil. For these tasks, human operators use all shovel faces, while autonomous systems so far are limited to excavation and dumping. Current methods often rely on heuristic models but do not incorporate soil mechanics. We address this shortcoming by using Reinforcement Learning in a GPU-parallelized Material Point Method particle simulation. Our controllers are conditioned on material state such as shape and compactness, enabling skills that use multiple contact faces of the tool and displace material both inside and outside of the shovel. To use the same learned weights across machines, our policies operate in a normalized end-effector space and are deployed through a calibrated machine interface. We evaluate this calibrated transfer on an 11.5t hydraulic excavator and a 500g tabletop robot. We validate performance through autonomous construction of a 42m long, 2.1m high embankment in 45min, executing 201 individual policy strokes without failure, retry, or operator intervention. In a direct comparison, the autonomous controller matches an expert operator's progression speed and produces a higher, more consistent embankment. Additional qualitative backfilling and compaction experiments demonstrate the material-state awareness and calibrated transfer across machines.