Logically-Constrained Neural Fitted Q-Iteration
Not on the map. Why?
Abstract (arXiv)
We propose a method for efficient training of Q-functions for continuous-state Markov Decision Processes (MDPs) such that the traces of the resulting policies satisfy a given Linear Temporal Logic (LTL) property. LTL, a modal logic, can express a wide range of time-dependent logical properties (including "safety") that are quite similar to patterns in natural language. We convert the LTL property into a limit deterministic Buchi automaton and construct an on-the-fly synchronised product MDP. The control policy is then synthesised by defining an adaptive reward function and by applying a modified neural fitted Q-iteration algorithm to the synchronised structure, assuming that no prior knowledge is available from the original MDP. The proposed method is evaluated in a numerical study to test the quality of the generated control policy and is compared with conventional methods for policy synthesis such as MDP abstraction (Voronoi quantizer) and approximate dynamic programming (fitted value iteration).
Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Sep 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2020: 2; 2021: 3; 2022: 5; 2023: 4; 2024: 2; 2025: 0; 2026: 1.
17 papers.
Sort:
By area:
- 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
- 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 39
- 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
- 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
- 2023ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learningcited by 15
- 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
- 2020Online Learning of Non-Markovian Reward Modelscited by 9
- 2021Programmatic Reward Design by Examplecited by 9
- 2022A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machinescited by 9
- 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectivescited by 9
- 2022Verified Probabilistic Policies for Deep Reinforcement Learningcited by 6
- 2022Reinforcement Learning Guided by Provable Normative Compliancecited by 3
5 more
- 2024Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verificationcited by 3
- 2023System III: Learning with Domain Knowledge for Safety Constraintscited by 2
- 2023Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesiscited by 1
- 2024Progressive Safeguards for Safe and Model-Agnostic Reinforcement Learningcited by 0
- 2026Value Functions as Supermartingale Certificatescited by 0
- 2026Value Functions as Supermartingale Certificates29 May 2026
- 2024Progressive Safeguards for Safe and Model-Agnostic Reinforcement Learning31 Oct 2024
- 2024Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification15 Jul 2024
- 2023Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis18 Dec 2023
- 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEval25 Sep 2023
- 2023ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learning24 Sep 2023
- 2023System III: Learning with Domain Knowledge for Safety Constraints23 Apr 2023
- 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning21 Sep 2022
- 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectives9 May 2022
- 2022A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machines20 Apr 2022
- 2022Reinforcement Learning Guided by Provable Normative Compliance30 Mar 2022
- 2022Verified Probabilistic Policies for Deep Reinforcement Learning10 Jan 2022
5 more
- 2021Programmatic Reward Design by Example14 Dec 2021
- 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic24 Feb 2021
- 2021Multi-Agent Reinforcement Learning with Temporal Logic Specifications1 Feb 2021
- 2020Online Learning of Non-Markovian Reward Models26 Sep 2020
- 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learning24 Jan 2020
- 2020Online Learning of Non-Markovian Reward Modelscited by 9
- 2020Online Learning of Non-Markovian Reward Models26 Sep 2020
- 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
- 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logiccited by 39
- 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
- 2023ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learningcited by 15
- 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
- 2021Programmatic Reward Design by Examplecited by 9
- 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectivescited by 9
- 2022Verified Probabilistic Policies for Deep Reinforcement Learningcited by 6
- 2022Reinforcement Learning Guided by Provable Normative Compliancecited by 3
- 2024Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verificationcited by 3
- 2023System III: Learning with Domain Knowledge for Safety Constraintscited by 2
- 2023Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesiscited by 1
12 most cited of 13 All 13 (JSON)
- 2026Value Functions as Supermartingale Certificates29 May 2026
- 2024Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification15 Jul 2024
- 2023Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesis18 Dec 2023
- 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEval25 Sep 2023
- 2023ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learning24 Sep 2023
- 2023System III: Learning with Domain Knowledge for Safety Constraints23 Apr 2023
- 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning21 Sep 2022
- 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectives9 May 2022
- 2022Reinforcement Learning Guided by Provable Normative Compliance30 Mar 2022
- 2022Verified Probabilistic Policies for Deep Reinforcement Learning10 Jan 2022
- 2021Programmatic Reward Design by Example14 Dec 2021
- 2021Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic24 Feb 2021
12 newest of 13 All 13 (JSON)