mlsanityData to 14 Oct 2026

Pareto Q-Learning with Reward Machines

arXiv 2606.19134 ↗First posted 17 Jun 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

We present Pareto Q-Learning with Reward Machines (PQLRM), a multi-objective reinforcement learning algorithm for tasks whose reward structure is specified by a set of reward machines (RMs). PQLRM combines Pareto Q-Learning (PQL), which maintains sets of vector-valued Q-estimates to approximate the Pareto front, with enhancements from Q-Learning with Reward Machines (QRM), which exploits the factored automaton structure of the reward signal. This yields a multi-policy algorithm that remains sample-efficient under non-Markovian, RM-encoded rewards. Experimental trials show that PQLRM converges faster than a naive PQL baseline applied to the cross-product MDP and can synthesize Pareto-optimal policies that QRM cannot.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 10 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

4 of 10 references.

References · 4

  1. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  2. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  3. 2021Energy-aware optimization of UAV base stations placement via decentralized multi-agent Q-learningcited by 3
  4. 2023Real Robot Challenge 2022: Learning Dexterous Manipulation from Offline Data in the Real Worldcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024An Empirical Investigation of Value-Based Multi-objective Reinforcement Learning for Stochastic Environments
  2. 2026Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning
  3. 2025Reinforcement Learning for Long-Horizon Unordered Tasks: From Boolean to Coupled Reward Machines
  4. 2025Model-Based Reinforcement Learning in Discrete-Action Non-Markovian Reward Decision Processes
  5. 2024On the Limitations of Markovian Rewards to Express Multi-Objective, Risk-Sensitive, and Modal Tasks
  6. 2025Learning Pareto-Optimal Rewards from Noisy Preferences: A Framework for Multi-Objective Inverse Reinforcement Learning
  7. 2025Reinforcement Learning with Stochastic Reward Machines
  8. 2025Discriminative Policy Optimization for Token-Level Reward Models