# DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planning

arXiv: https://arxiv.org/abs/2410.17186 · HTML: https://mlsanity.com/paper/2410.17186/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations](https://mlsanity.com/paper/2003.08938/) (2020)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Robust Deep Reinforcement Learning with Adversarial Attacks](https://mlsanity.com/paper/1712.03632/) (2017)
- [Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2005.06800/) (2020)
- [Robust Reinforcement Learning for Continuous Control with Model Misspecification](https://mlsanity.com/paper/1906.07516/) (2019)
- [Soft-Robust Actor-Critic Policy-Gradient](https://mlsanity.com/paper/1803.04848/) (2018)
- [Robust Imitation Learning against Variations in Environment Dynamics](https://mlsanity.com/paper/2206.09314/) (2022)
- [An informative path planning framework for UAV-based terrain monitoring](https://mlsanity.com/paper/1809.03870/) (2018)
- [Domain Randomization via Entropy Maximization](https://mlsanity.com/paper/2311.01885/) (2023)
- [Learning Robust Options](https://mlsanity.com/paper/1802.03236/) (2018)
- [Deep Reinforcement Learning-based Large-scale Robot Exploration](https://mlsanity.com/paper/2403.10833/) (2024)
- [Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing](https://mlsanity.com/paper/2109.13570/) (2021)
- [Robust Visual Domain Randomization for Reinforcement Learning](https://mlsanity.com/paper/1910.10537/) (2019)
- [Deep Reinforcement Learning with Dynamic Graphs for Adaptive Informative Path Planning](https://mlsanity.com/paper/2402.04894/) (2024)
- [Informative Path Planning for Mobile Sensing with Reinforcement Learning](https://mlsanity.com/paper/2002.07890/) (2020)
- [FireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams](https://mlsanity.com/paper/2011.00165/) (2020)
- [Spatio-Temporal Attention Network for Persistent Monitoring of Multiple Mobile Targets](https://mlsanity.com/paper/2303.06350/) (2023)
- [Multi-Robot Informative Path Planning from Regression with Sparse Gaussian Processes (with Appendix)](https://mlsanity.com/paper/2309.07050/) (2023)
- [Adaptive Selection of Informative Path Planning Strategies via Reinforcement Learning](https://mlsanity.com/paper/2108.06618/) (2021)
- [OffRIPP: Offline RL-based Informative Path Planning](https://mlsanity.com/paper/2409.16830/) (2024)

## Abstract (arXiv)

Informative path planning (IPP) is an important planning paradigm for various real-world robotic applications such as environment monitoring. IPP involves planning a path that can learn an accurate belief of the quantity of interest, while adhering to planning constraints. Traditional IPP methods typically require high computation time during execution, giving rise to reinforcement learning (RL) based IPP methods. However, the existing RL-based methods do not consider spatio-temporal environments which involve their own challenges due to variations in environment characteristics. In this paper, we propose DyPNIPP, a robust RL-based IPP framework, designed to operate effectively across spatio-temporal environments with varying dynamics. To achieve this, DyPNIPP incorporates domain randomization to train the agent across diverse environments and introduces a dynamics prediction model to capture and adapt the agent actions to specific environment dynamics. Our extensive experiments in a wildfire environment demonstrate that DyPNIPP outperforms existing RL-based IPP algorithms by significantly improving robustness and performing across diverse environment conditions.