mlsanityData to 14 Oct 2026

Physics-Informed Reward Machines

arXiv 2508.14093 ↗First posted 14 Aug 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Reward machines (RMs) provide a structured way to specify non-Markovian rewards in reinforcement learning (RL), thereby improving both expressiveness and programmability. Viewed more broadly, they separate what is known about the environment, captured by the reward mechanism, from what remains unknown and must be discovered through sampling. This separation supports techniques such as counterfactual experience generation and reward shaping, which reduce sample complexity and speed up learning. We introduce physics-informed reward machines (pRMs), a symbolic machine designed to express complex learning objectives and reward structures for RL agents, thereby enabling more programmable, expressive, and efficient learning. We present RL algorithms capable of exploiting pRMs via counterfactual experiences and reward shaping. Our experimental results show that these techniques accelerate reward acquisition during the training phases of RL. We demonstrate the expressiveness and effectiveness of pRMs through experiments in both finite and continuous physical environments, illustrating that incorporating pRMs significantly improves learning efficiency across several control tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 34 of its 62 references that are in the map. First posted on arXiv 14 Aug 2025.

Builds on

34 of 62 references.

References · 34

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  3. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  4. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
  5. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  6. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  7. 2020Exploration-Exploitation in Constrained MDPscited by 149
  8. 2017Machine Learning of Linear Differential Equations using Gaussian Processescited by 140
  9. 2019Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guaranteescited by 77
  10. 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 75
  11. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 75
  12. 2019Joint Inference of Reward Machines and Policies for Reinforcement Learningcited by 56
22 more
  1. 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 54
  2. 2014A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specificationscited by 49
  3. 2014Verification of Markov Decision Processes using Learning Algorithmscited by 39
  4. 2016Inferring solutions of differential equations using noisy multi-fidelity datacited by 38
  5. 2019Certified Reinforcement Learning with Logic Guidancecited by 36
  6. 2020Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Taskscited by 28
  7. 2020Induction and Exploitation of Subgoal Automata for Reinforcement Learningcited by 26
  8. 2023A Survey on Physics Informed Reinforcement Learning: Review and Open Problemscited by 24
  9. 2020Active Finite Reward Automaton Inference and Reinforcement Learning Using Queries and Counterexamplescited by 23
  10. 2022Physics-Informed Model-Based Reinforcement Learningcited by 18
  11. 2019Induction of Subgoal Automata for Reinforcement Learningcited by 17
  12. 2020Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learningcited by 15
  13. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  14. 2021Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learningcited by 13
  15. 2020Learning Non-Markovian Reward Models in MDPscited by 11
  16. 2020AMYTISS: Parallelized Automated Controller Synthesis for Large-Scale Stochastic Systemscited by 8
  17. 2023Bridging Physics-Informed Neural Networks with Reinforcement Learning: Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO)cited by 7
  18. 2024Efficient Reinforcement Learning in Probabilistic Reward Machinescited by 4
  19. 2022Compositional Reinforcement Learning for Discrete-Time Stochastic Control Systemscited by 2
  20. 2022PhysQ: A Physics Informed Reinforcement Learning Framework for Building Controlcited by 2
  21. 2018Fast Online Exact Solutions for Deterministic MDPs with Sparse Rewardscited by 1
  22. 2024Data-driven Construction of Finite Abstractions for Interconnected Systems: A Compositional Approachcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcement Learning with Symbolic Reward Machines
  2. 2024Reward Machines for Deep RL in Noisy and Uncertain Environments
  3. 2025Pushdown Reward Machines for Reinforcement Learning
  4. 2024Neural Reward Machines
  5. 2025Expressive Reward Synthesis with the Runtime Monitoring Language
  6. 2025Fully Learnable Neural Reward Machines
  7. 2025About Time: Model-free Reinforcement Learning with Timed Reward Machines
  8. 2022A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machines