# Residual Reinforcement Learning for Waste-Container Lifting Using Large-Scale Cranes with Underactuated Tools

arXiv: https://arxiv.org/abs/2602.05895 · HTML: https://mlsanity.com/paper/2602.05895/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 13 of 13)

- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Residual Off-Policy RL for Finetuning Behavior Cloning Policies](https://mlsanity.com/paper/2509.19301/) (2025)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)
- [Residual Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2106.08050/) (2021)
- [Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning](https://mlsanity.com/paper/2511.04831/) (2025)
- [Reinforcement Learning Control of a Forestry Crane Manipulator](https://mlsanity.com/paper/2103.02315/) (2021)
- [HEAP -- The autonomous walking excavator](https://mlsanity.com/paper/2106.05059/) (2021)
- [Reinforcement Learning Control for Autonomous Hydraulic Material Handling Machines with Underactuated Tools](https://mlsanity.com/paper/2410.05093/) (2024)
- [Dynamic Throwing with Robotic Material Handling Machines](https://mlsanity.com/paper/2405.19001/) (2024)
- [Large Scale Robotic Material Handling: Learning, Planning, and Control](https://mlsanity.com/paper/2508.09003/) (2025)
- [Towards Learning Boulder Excavation with Hydraulic Excavators](https://mlsanity.com/paper/2509.17683/) (2025)
- [Multi-log grasping using reinforcement learning and virtual visual servoing](https://mlsanity.com/paper/2309.02997/) (2023)
- [ExT: Towards Scalable Autonomous Excavation via Large-Scale Multi-Task Pretraining and Fine-Tuning](https://mlsanity.com/paper/2509.14992/) (2025)

## Abstract (arXiv)

This paper studies the container lifting phase of a waste-container recycling task in urban environments, performed by a hydraulic loader crane equipped with an underactuated discharge unit, and proposes a residual reinforcement learning (RRL) approach that combines a nominal Cartesian controller with a learned residual policy. All experiments are conducted in simulation, where the task is characterized by tight geometric tolerances between the discharge-unit hooks and the container rings relative to the overall crane scale, making precise trajectory tracking and swing suppression essential. The nominal controller uses admittance control for trajectory tracking and pendulum-aware swing damping, followed by damped least-squares inverse kinematics with a nullspace posture term to generate joint velocity commands. A PPO-trained residual policy in Isaac Lab compensates for unmodeled dynamics and parameter variations, improving precision and robustness without requiring end-to-end learning from scratch. We further employ randomized episode initialization and domain randomization over payload properties, actuator gains, and passive joint parameters to enhance generalization. Simulation results demonstrate improved tracking accuracy, reduced oscillations, and higher lifting success rates compared to the nominal controller alone.