Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specifications
On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL
Abstract (arXiv)
We present a novel end-to-end model-based Reinforcement Learning (RL) algorithm for efficient policy synthesis under given Linear Temporal Logic (LTL) specifications (e.g., safety or reachability) in unknown environments. To do so, a Limit-Deterministic B"uchi Automaton (LDBA) representation of the LTL task is synchronised with a Bayes-Adaptive Markov Decision Process (BAMDP) representation of the environment, which allows us to leverage an enhanced exploration-exploitation trade-off that is achieved via Bayesian RL, as opposed to traditional non-Bayesian approaches. We further propose a novel Bayes-Adaptive Monte-Carlo Planning (BAMCP) algorithm to allow for approximate Bayes-optimal strategy synthesis in the synchronised BAMDP construct. A range of finite- and infinite-horizon task experiments demonstrate the effectiveness of our approach in terms of both property satisfaction and sample efficiency, when compared to traditional model-free approaches. Additional ablation studies also successfully highlight the value of the novel BAMCP algorithm in comparison to classical BAMCP for LTL task satisfaction. Finally, we also showcase a successful application of our approach for textitcautious RL, namely to reduce the number of task violations incurred during policy training.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 38 references that are in the map. First posted on arXiv 17 Sep 2026.
Builds on
18 of 38 references.
Closest ancestors · 1
Same direction, newest first
Other references · 17
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2016OpenAI Gymcited by 1,736
- 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
- 2019Meta reinforcement learning as task inferencecited by 77
- 2018Omega-Regular Objectives in Model-Free Reinforcement Learningcited by 75
- 2018Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specificationscited by 72
- 2018Logically-Constrained Reinforcement Learningcited by 51
- 2020Cautious Reinforcement Learning with Logical Constraintscited by 45
- 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 39
- 2019Certified Reinforcement Learning with Logic Guidancecited by 36
- 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 29
5 more
- 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
- 2022Policy Optimization with Linear Temporal Logic Constraintscited by 19
- 2024Bayes Adaptive Monte Carlo Tree Search for Offline Model-based Reinforcement Learningcited by 11
- 2021Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specificationscited by 5
- 2021Reinforcement Learning with Temporal Logic Constraints for Partially-Observable Markov Decision Processescited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Exploration
- 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logic
- 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulation
- 2024Regret-Free Reinforcement Learning for LTL Specifications
- 2024When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL
- 2025Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM Reasoning
- 2024Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayes Theory
- 2025Learning Probabilistic Temporal Logic Specifications for Stochastic Systems