mlsanityData to 14 Oct 2026

Learning From Failures: Efficient Reinforcement Learning Control with Episodic Memory

arXiv 2603.07110 ↗First posted 7 Mar 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning has achieved remarkable success in robot learning. However, under challenging exploration and contact-rich dynamics, early-stage training is frequently dominated by premature terminations such as collisions and falls. As a result, learning is overwhelmed by short-horizon, low-return trajectories, which hinder convergence and limit long-horizon exploration. To alleviate this issue, we propose a technique called Failure Episodic Memory Alert (FEMA). FEMA explicitly stores short-horizon failure experiences through an episodic memory module. During interactions, it retrieves similar failure experiences and prevents the robot from recurrently relapsing into unstable states, guiding the policy toward long-horizon trajectories with greater long-term value. FEMA can be combined easily with model-free reinforcement learning algorithms, and yields a substantial sample-efficiency improvement of 33.11% on MuJoCo tasks across several classical RL algorithms. Furthermore, integrating FEMA into a parallelized PPO training pipeline demonstrates its effectiveness on a real-world bipedal robot task.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 30 references that are in the map. First posted on arXiv 7 Mar 2026.

Builds on

22 of 30 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  6. 2024Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learningcited by 189
  7. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  8. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 175
  9. 2018Learning to Walk via Deep Reinforcement Learningcited by 158
  10. 2017Neural Episodic Controlcited by 151
  11. 2016Model-Free Episodic Controlcited by 102
  12. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
10 more
  1. 2023DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learningcited by 66
  2. 2018Episodic Memory Deep Q-Networkscited by 42
  3. 2019Learning Robotic Manipulation through Visual Planning and Actingcited by 41
  4. 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 32
  5. 2021Generalizable Episodic Memory for Deep Reinforcement Learningcited by 23
  6. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 21
  7. 2022Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interactioncited by 17
  8. 2019Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environmentscited by 14
  9. 2021Solving Continuous Control with Episodic Memorycited by 8
  10. 2024Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Failure-Aware RL: Reliable Offline-to-Online Reinforcement Learning with Self-Recovery for Real-World Manipulation
  2. 2026Learning from Demonstration with Failure Awareness for Safe Robot Navigation
  3. 2026FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement
  4. 2025Symmetry-Guided Memory Augmentation for Efficient Locomotion Learning
  5. 2026OnEvoMemory: Evolving Memory through Online Robot Rollouts for Pretrained Robot Policies ↗
  6. 2026Safe Reinforcement Learning with Receding-Horizon Safety Critic and Differentiable Action Shields
  7. 2026Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents
  8. 2026Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures