# MoRe-ERL: Learning Motion Residuals using Episodic Reinforcement Learning

arXiv: https://arxiv.org/abs/2508.01409 · HTML: https://mlsanity.com/paper/2508.01409/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 2 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [Motion Policy Networks](https://mlsanity.com/paper/2210.12209/) (2022)
- [Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulation](https://mlsanity.com/paper/2008.07682/) (2020)
- [ProDMPs: A Unified Perspective on Dynamic and Probabilistic Movement Primitives](https://mlsanity.com/paper/2210.01531/) (2022)
- [Deep Black-Box Reinforcement Learning with Movement Primitives](https://mlsanity.com/paper/2210.09622/) (2022)
- [Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learning](https://mlsanity.com/paper/2401.11437/) (2024)
- [Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertainty](https://mlsanity.com/paper/2106.04306/) (2021)
- [MP3: Movement Primitive-Based (Re-)Planning Policy](https://mlsanity.com/paper/2306.12729/) (2023)
- [ST-RRT*: Asymptotically-Optimal Bidirectional Motion Planning through Space-Time](https://mlsanity.com/paper/2203.02176/) (2022)
- [Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning](https://mlsanity.com/paper/2408.14063/) (2024)
- [BMP: Bridging the Gap between B-Spline and Movement Primitives](https://mlsanity.com/paper/2411.10336/) (2024)
- [Planning with Learned Subgoals Selected by Temporal Information](https://mlsanity.com/paper/2410.20272/) (2024)
- [HIRO: Heuristics Informed Robot Online Path Planning Using Pre-computed Deterministic Roadmaps](https://mlsanity.com/paper/2410.20279/) (2024)

## Abstract (arXiv)

We propose MoRe-ERL, a framework that combines Episodic Reinforcement Learning (ERL) and residual learning, which refines preplanned reference trajectories into safe, feasible, and efficient task-specific trajectories. This framework is general enough to incorporate into arbitrary ERL methods and motion generators seamlessly. MoRe-ERL identifies trajectory segments requiring modification while preserving critical task-related maneuvers. Then it generates smooth residual adjustments using B-Spline-based movement primitives to ensure adaptability to dynamic task contexts and smoothness in trajectory refinement. Experimental results demonstrate that residual learning significantly outperforms training from scratch using ERL methods, achieving superior sample efficiency and task performance. Hardware evaluations further validate the framework, showing that policies trained in simulation can be directly deployed in real-world systems, exhibiting a minimal sim-to-real gap.