# RAPid-Learn: A Framework for Learning to Recover for Handling Novelties in Open-World Environments

arXiv: https://arxiv.org/abs/2206.12493 · HTML: https://mlsanity.com/paper/2206.12493/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 24 Jun 2022.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning](https://mlsanity.com/paper/1811.00090/) (2018)
- [Reinforcement Learning for Non-Stationary Markov Decision Processes: The Blessing of (More) Optimism](https://mlsanity.com/paper/2006.14389/) (2020)
- [Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems](https://mlsanity.com/paper/2102.07659/) (2021)
- [Lipschitz Lifelong Reinforcement Learning](https://mlsanity.com/paper/2001.05411/) (2020)
- [SPOTTER: Extending Symbolic Planning Operators through Targeted Reinforcement Learning](https://mlsanity.com/paper/2012.13037/) (2020)
- [Reinforcement Learning in Non-Stationary Environments](https://mlsanity.com/paper/1905.03970/) (2019)
- [Minimalistic Attacks: How Little it Takes to Fool a Deep Reinforcement Learning Policy](https://mlsanity.com/paper/1911.03849/) (2019)
- [Tool Macgyvering: Tool Construction Using Geometric Reasoning](https://mlsanity.com/paper/1902.03666/) (2019)
- [L2Explorer: A Lifelong Reinforcement Learning Assessment Environment](https://mlsanity.com/paper/2203.07454/) (2022)
- [From semantics to execution: Integrating action planning with reinforcement learning for robotic causal problem-solving](https://mlsanity.com/paper/1905.09683/) (2019)
- [Learning to Navigate Cloth using Haptics](https://mlsanity.com/paper/1703.06905/) (2017)

## Built on by (most cited first, 13 of 13)

- [NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds](https://mlsanity.com/paper/2401.03546/) (2024)
- [Curiosity-Driven Imagination: Discovering Plan Operators and Learning Associated Policies for Open-World Adaptation](https://mlsanity.com/paper/2503.04931/) (2025)
- [SLAP: Shortcut Learning for Abstract Planning](https://mlsanity.com/paper/2511.01107/) (2025)
- [Graph Pruning for Enumeration of Minimal Unsatisfiable Subsets](https://mlsanity.com/paper/2402.15524/) (2024)
- [Few-Shot Neuro-Symbolic Imitation Learning for Long-Horizon Planning and Acting](https://mlsanity.com/paper/2508.21501/) (2025)
- [Heterogeneous Knowledge for Augmented Modular Reinforcement Learning](https://mlsanity.com/paper/2306.01158/) (2023)
- [Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery](https://mlsanity.com/paper/2311.14270/) (2023)
- [Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings](https://mlsanity.com/paper/2402.08145/) (2024)
- [Learning to Bridge the Gap: Efficient Novelty Recovery with Planning and Reinforcement Learning](https://mlsanity.com/paper/2409.19226/) (2024)
- [Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning](https://mlsanity.com/paper/2603.11351/) (2026)
- [Rapid Open-World Adaptation by Adaptation Principles Learning](https://mlsanity.com/paper/2312.11138/) (2023)
- [Breaking Task Impasses Quickly: Adaptive Neuro-Symbolic Learning for Open-World Robotics](https://mlsanity.com/paper/2601.16985/) (2026)
- [Build on Priors: Vision--Language--Guided Neuro-Symbolic Imitation Learning for Data-Efficient Real-World Robot Manipulation](https://mlsanity.com/paper/2604.03759/) (2026)

## Abstract (arXiv)

We propose RAPid-Learn: Learning to Recover and Plan Again, a hybrid planning and learning method, to tackle the problem of adapting to sudden and unexpected changes in an agent's environment (i.e., novelties). RAPid-Learn is designed to formulate and solve modifications to a task's Markov Decision Process (MDPs) on-the-fly and is capable of exploiting domain knowledge to learn any new dynamics caused by the environmental changes. It is capable of exploiting the domain knowledge to learn action executors which can be further used to resolve execution impasses, leading to a successful plan execution. This novelty information is reflected in its updated domain model. We demonstrate its efficacy by introducing a wide variety of novelties in a gridworld environment inspired by Minecraft, and compare our algorithm with transfer learning baselines from the literature. Our method is (1) effective even in the presence of multiple novelties, (2) more sample efficient than transfer learning RL baselines, and (3) robust to incomplete model information, as opposed to pure symbolic planning approaches.