mlsanityData to 14 Oct 2026

Learning Excavation of Rigid Objects with Offline Reinforcement Learning

arXiv 2303.16427 ↗First posted 29 Mar 2023cs.ROcited by 2

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Autonomous excavation is a challenging task. The unknown contact dynamics between the excavator bucket and the terrain could easily result in large contact forces and jamming problems during excavation. Traditional model-based methods struggle to handle such problems due to complex dynamic modeling. In this paper, we formulate the excavation skills with three novel manipulation primitives. We propose to learn the manipulation primitives with offline reinforcement learning (RL) to avoid large amounts of online robot interactions. The proposed method can learn efficient penetration skills from sub-optimal demonstrations, which contain sub-trajectories that can be ``stitched" together to formulate an optimal trajectory without causing jamming. We evaluate the proposed method with extensive experiments on excavating a variety of rigid objects and demonstrate that the learned policy outperforms the demonstrations. We also show that the learned policy can quickly adapt to unseen and challenging fragmented rocks with online fine-tuning.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 34 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

16 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Offline Meta-Reinforcement Learning for Industrial Insertion8 Oct

Other references · 15

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  3. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  4. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  5. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  6. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  7. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  8. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 115
  9. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 86
  10. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 53
  11. 2021Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Taskscited by 20
  12. 2022Excavation Reinforcement Learning Using Geometric Representationcited by 7
3 more
  1. 2020Optimization-Based Framework for Excavation Trajectory Generationcited by 6
  2. 2021Excavation Learning for Rigid Objects in Cluttercited by 4
  3. 2021Dynamic Modeling of Bucket-Soil Interactions Using Koopman-DFL Lifting Linearization for Model Predictive Contouring Control of Autonomous Excavatorscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024ExACT: An End-to-End Autonomous Excavator System Using Action Chunking With Transformerscited by 4
  2. 2025Interactive Shaping of Granular Media Using Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Learning Boulder Excavation with Hydraulic Excavators
  2. 2025Learning to Capture Rocks using an Excavator: A Reinforcement Learning Approach with Guiding Reward Formulation
  3. 2026Size Doesn't Matter: Material-State Reinforcement Learning for Excavator Transferable Soil Manipulation
  4. 2025ExT: Towards Scalable Autonomous Excavation via Large-Scale Multi-Task Pretraining and Fine-Tuning
  5. 2025Learning Tool-Aware Adaptive Compliant Control for Autonomous Regolith Excavation
  6. 20243D Operation of Autonomous Excavator based on Reinforcement Learning through Independent Reward for Individual Joints
  7. 2024Progressive-Resolution Policy Distillation: Leveraging Coarse-Resolution Simulations for Time-Efficient Fine-Resolution Policy Learning
  8. 2023Offline Skill Generalization via Task and Motion Planning