# A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

arXiv: https://arxiv.org/abs/2309.01909 · HTML: https://mlsanity.com/paper/2309.01909/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 87 of its 172 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 30 of 87)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)
- [First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations](https://mlsanity.com/paper/1704.02463/) (2017)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances](https://mlsanity.com/paper/1911.10868/) (2019)
- [Entity Abstraction in Visual Model-Based Reinforcement Learning](https://mlsanity.com/paper/1910.12827/) (2019)
- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [Model-free Deep Reinforcement Learning for Urban Autonomous Driving](https://mlsanity.com/paper/1904.09503/) (2019)
- [Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2005.06800/) (2020)

## Built on by (most cited first, 24 of 24)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [A Mass-Conserving-Perceptron for Machine Learning-Based Modeling of Geoscientific Systems](https://mlsanity.com/paper/2310.08644/) (2023)
- [Adapting Physics-Informed Neural Networks to Improve ODE Optimization in Mosquito Population Dynamics](https://mlsanity.com/paper/2406.05108/) (2024)
- [Self-Evolving Scientific Agent Designs Physically Reasoned White-Box Fluid Control](https://mlsanity.com/paper/2606.08405/) (2026)
- [HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems](https://mlsanity.com/paper/2501.04538/) (2025)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [ResWM: Residual-Action World Model for Visual RL](https://mlsanity.com/paper/2603.11110/) (2026)
- [Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes](https://mlsanity.com/paper/2402.11179/) (2024)
- [Link Prediction with Physics-Inspired Graph Neural Networks](https://mlsanity.com/paper/2402.14802/) (2024)
- [Physics-informed RL for Maximal Safety Probability Estimation](https://mlsanity.com/paper/2403.16391/) (2024)
- [Physics-Driven Local-Whole Elastic Deformation Modeling for Point Cloud Representation Learning](https://mlsanity.com/paper/2505.13812/) (2025)
- [Articulated-Body Dynamics Network: Dynamics-Grounded Prior for Robot Learning](https://mlsanity.com/paper/2603.19078/) (2026)
- [Coordinated Power Smoothing Control for Wind Storage Integrated System with Physics-informed Deep Reinforcement Learning](https://mlsanity.com/paper/2412.17838/) (2024)
- [SALSA-RL: Stability Analysis in the Latent Space of Actions for Reinforcement Learning](https://mlsanity.com/paper/2502.15512/) (2025)
- [Physics-Informed Reward Machines](https://mlsanity.com/paper/2508.14093/) (2025)
- [Physics-Informed Operator Learning for Hemodynamic Modeling](https://mlsanity.com/paper/2509.17293/) (2025)
- [Control of dynamical systems with neural networks](https://mlsanity.com/paper/2510.12810/) (2025)
- [Fundamentals of Physical AI](https://mlsanity.com/paper/2511.09497/) (2025)
- [The Enhanced Physics-Informed Kolmogorov-Arnold Networks: Applications of Newton's Laws in Financial Deep Reinforcement Learning (RL) Algorithms](https://mlsanity.com/paper/2602.01388/) (2026)
- [Co-Learning Port-Hamiltonian Systems and Optimal Energy-Shaping Control](https://mlsanity.com/paper/2604.26172/) (2026)
- [Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems](https://mlsanity.com/paper/2607.16177/) (2026)
- [Adaptive Reconstruction of Bosonic Quantum States](https://mlsanity.com/paper/2608.02049/) (2026)
- [Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis](https://mlsanity.com/paper/2608.17117/) (2026)
- [Seven Sources of Physical AI Capability Formation](https://mlsanity.com/paper/2609.09627/) (2026)

## Abstract (arXiv)

The inclusion of physical information in machine learning frameworks has revolutionized many application areas. This involves enhancing the learning process by incorporating physical constraints and adhering to physical laws. In this work we explore their utility for reinforcement learning applications. We present a thorough review of the literature on incorporating physics information, as known as physics priors, in reinforcement learning approaches, commonly referred to as physics-informed reinforcement learning (PIRL). We introduce a novel taxonomy with the reinforcement learning pipeline as the backbone to classify existing works, compare and contrast them, and derive crucial insights. Existing works are analyzed with regard to the representation/ form of the governing physics modeled for integration, their specific contribution to the typical reinforcement learning architecture, and their connection to the underlying reinforcement learning pipeline stages. We also identify core learning architectures and physics incorporation biases (i.e., observational, inductive and learning) of existing PIRL approaches and use them to further categorize the works for better understanding and adaptation. By providing a comprehensive perspective on the implementation of the physics-informed capability, the taxonomy presents a cohesive approach to PIRL. It identifies the areas where this approach has been applied, as well as the gaps and opportunities that exist. Additionally, the taxonomy sheds light on unresolved issues and challenges, which can guide future research. This nascent field holds great potential for enhancing reinforcement learning algorithms by increasing their physical plausibility, precision, data efficiency, and applicability in real-world scenarios.