mlsanityData to 7 Oct 2026

Sample-efficient Neuro-symbolic Proximal Policy Optimization

arXiv 2604.25534 ↗First posted 28 Apr 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning (DRL) algorithms often require a large amount of data and struggle in sparse-reward domains with long planning horizons and multiple sub-goals. In this paper, we propose a neuro-symbolic extension of Proximal Policy Optimization (PPO) that transfers partial logical policy specifications learned in easier instances to guide learning in more challenging settings. We introduce two integrations of symbolic guidance: (i) H-PPO-Product, which biases the action distribution at sampling time, and (ii) H-PPO-SymLoss, which augments the PPO loss with a symbolic regularization term. We evaluate our methods on three benchmarks (OfficeWorld, WaterWorld, and DoorKey), showing consistently faster learning and higher return at convergence than PPO and a Reward Machine baseline, also under imperfect symbolic knowledge.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

20 of 29 references.

References · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 320
  4. 2017A Semantic Loss Function for Deep Learning with Symbolic Knowledgecited by 267
  5. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  6. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 183
  7. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  8. 2020Phasic Policy Gradientcited by 121
  9. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  10. 2021From Statistical Relational to Neurosymbolic Artificial Intelligence: a Surveycited by 72
  11. 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 69
  12. 2021Heuristic-Guided Reinforcement Learningcited by 41
8 more
  1. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  2. 2020Induction and Exploitation of Subgoal Automata for Reinforcement Learningcited by 26
  3. 2019On the Generalization Gap in Reparameterizable Reinforcement Learningcited by 21
  4. 2023Deep Explainable Relational Reinforcement Learning: A Neuro-Symbolic Approachcited by 15
  5. 2024Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approachcited by 7
  6. 2025Online inductive learning from answer sets for efficient reinforcement learning explorationcited by 4
  7. 2025Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Timecited by 2
  8. 2026Sample-Efficient Neurosymbolic Deep Reinforcement Learningcited by 1

Built on by

Not cited yet.