mlsanityData to 7 Oct 2026

Foundations for Restraining Bolts: Reinforcement Learning with LTLf/LDLf restraining specifications

arXiv 1807.06333 ↗First posted 17 Jul 2018cs.LG cs.AI stat.MLcited by 69

Not on the map. Why?

Abstract (arXiv)

In this work we investigate on the concept of "restraining bolt", envisioned in Science Fiction. Specifically we introduce a novel problem in AI. We have two distinct sets of features extracted from the world, one by the agent and one by the authority imposing restraining specifications (the "restraining bolt"). The two sets are apparently unrelated since of interest to independent parties, however they both account for (aspects of) the same world. We consider the case in which the agent is a reinforcement learning agent on the first set of features, while the restraining bolt is specified logically using linear time logic on finite traces LTLf/LDLf over the second set of features. We show formally, and illustrate with examples, that, under general circumstances, the agent can learn while shaping its goals to suitably conform (as much as possible) to the restraining bolt specifications.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 10; 2022: 14; 2023: 12; 2024: 10; 2025: 8; 2026: 13.

69 papers.

Sort:

By area:

  1. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  2. 2023Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environmentscited by 52
  3. 2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 41
  4. 2021Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systemscited by 36
  5. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 31
  6. 2021The Logical Options Frameworkcited by 24
  7. 2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
  8. 2024SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Modelscited by 21
  9. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  10. 2021On the (In)Tractability of Reinforcement Learning for LTL Objectivescited by 18
  11. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 17
  12. 2022Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environmentscited by 16
38 more
  1. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  2. 2023Computably Continuous Reinforcement-Learning Objectives are PAC-learnablecited by 13
  3. 2020Learning Non-Markovian Reward Models in MDPscited by 11
  4. 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
  5. 2023LgTS: Dynamic Task Sampling using LLM-generated sub-goals for Reinforcement Learning Agentscited by 11
  6. 2022Markov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processescited by 10
  7. 2020Online Learning of Non-Markovian Reward Modelscited by 9
  8. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
  9. 2023Learning Logic Specifications for Soft Policy Guidance in POMCPcited by 7
  10. 2023Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approachcited by 7
  11. 2023Controlled Diversity with Preference : Towards Learning a Diverse Set of Desired Skillscited by 6
  12. 2024Neural Reward Machinescited by 6
  13. 2021Target Languages (vs. Inductive Biases) for Learning to Act and Plancited by 5
  14. 2021Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machinescited by 5
  15. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  16. 2024Adaptive Reward Design for Reinforcement Learningcited by 5
  17. 2024Guiding Multi-agent Multi-task Reinforcement Learning by a Hierarchical Framework with Logical Reward Shapingcited by 4
  18. 2025Online inductive learning from answer sets for efficient reinforcement learning explorationcited by 4
  19. 2022Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Drivingcited by 3
  20. 2023Model Checking Strategies from Synthesis Over Finite Tracescited by 3
  21. 2024Inductive Generalization in Reinforcement Learning from Specificationscited by 3
  22. 2021LTL-Constrained Steady-State Policy Synthesiscited by 2
  23. 2021Lifted Model Checking for Relational MDPscited by 2
  24. 2022Planning for Temporally Extended Goals in Pure-Past Linear Temporal Logic: A Polynomial Reduction to Standard Planningcited by 2
  25. 2024Ensuring trustworthy and ethical behaviour in intelligent logical agentscited by 2
  26. 2024Tractable Offline Learning of Regular Decision Processescited by 2
  27. 2025HypRL: Reinforcement Learning of Control Policies for Hyperpropertiescited by 2
  28. 2025Learning Symbolic Persistent Macro-Actions for POMDP Solving Over Timecited by 2
  29. 2021Collaborative Human-Agent Planning for Resiliencecited by 1
  30. 2021Regular Decision Processes for Grid Worldscited by 1
  31. 2022Learning to act: a Reinforcement Learning approach to recommend the best next activitiescited by 1
  32. 2022Automata Cascades: Expressivity and Sample Complexitycited by 1
  33. 2023Safeguarded Progress in Reinforcement Learning: Safe Bayesian Exploration for Control Policy Synthesiscited by 1
  34. 2024Learning Hidden Subgoals under Temporal Ordering Constraints in Reinforcement Learningcited by 1
  35. 2025Fully Learnable Neural Reward Machinescited by 1
  36. 2026Sample-Efficient Neurosymbolic Deep Reinforcement Learningcited by 1
  37. 2026Grounding LTL Tasks in Sub-Symbolic RL Environments for Zero-Shot Generalizationcited by 1
  38. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1

50 most cited of 69 All 69 (JSON)