# Learning Minimally-Violating Continuous Control for Infeasible Linear Temporal Logic Specifications

arXiv: https://arxiv.org/abs/2210.01162 · HTML: https://mlsanity.com/paper/2210.01162/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 49 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.04717/) (2016)
- [Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning](https://mlsanity.com/paper/2010.03950/) (2020)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees](https://mlsanity.com/paper/1909.05304/) (2019)
- [Learning Navigation Behaviors End-to-End with AutoRL](https://mlsanity.com/paper/1809.10124/) (2018)
- [Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?](https://mlsanity.com/paper/1909.10618/) (2019)
- [Joint Inference of Reward Machines and Policies for Reinforcement Learning](https://mlsanity.com/paper/1909.05912/) (2019)
- [A Learning Based Approach to Control Synthesis of Markov Decision Processes for Linear Temporal Logic Specifications](https://mlsanity.com/paper/1409.5486/) (2014)
- [Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Design](https://mlsanity.com/paper/1503.03144/) (2015)
- [Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic](https://mlsanity.com/paper/2102.12855/) (2021)
- [Reinforcement Learning Based Temporal Logic Control with Maximum Probabilistic Satisfaction](https://mlsanity.com/paper/2010.06797/) (2020)
- [Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewards](https://mlsanity.com/paper/2203.10229/) (2022)
- [FaSTrack: a Modular Framework for Real-Time Motion Planning and Guaranteed Safe Tracking](https://mlsanity.com/paper/2102.07039/) (2021)
- [Optimal Probabilistic Motion Planning with Potential Infeasible LTL Constraints](https://mlsanity.com/paper/2007.14325/) (2020)
- [Neural Network Verification in Control](https://mlsanity.com/paper/2110.01388/) (2021)
- [Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications](https://mlsanity.com/paper/2201.12231/) (2022)
- [An Abstraction-Free Method for Multi-Robot Temporal Logic Optimal Control Synthesis](https://mlsanity.com/paper/1909.00526/) (2019)
- [Accelerated Reinforcement Learning for Temporal Logic Control Objectives](https://mlsanity.com/paper/2205.04424/) (2022)
- [Model Predictive Control for Signal Temporal Logic Specification](https://mlsanity.com/paper/1703.09563/) (2017)
- [What to Do When You Can't Do It All: Temporal Logic Planning with Soft Temporal Logic Constraints](https://mlsanity.com/paper/2008.01926/) (2020)
- [Minimum-Violation Planning for Autonomous Systems: Theoretical and Practical Considerations](https://mlsanity.com/paper/2009.11954/) (2020)
- [Receding Horizon Control Based Online Motion Planning with Partially Infeasible LTL Specifications](https://mlsanity.com/paper/2007.12123/) (2020)
- [Automata-based Optimal Planning with Relaxed Specifications](https://mlsanity.com/paper/2107.13650/) (2021)
- [On the Minimal Revision Problem of Specification Automata](https://mlsanity.com/paper/1404.2289/) (2014)
- [Minimum Violation Control Synthesis on Cyber-Physical Systems under Attacks](https://mlsanity.com/paper/1809.00975/) (2018)

## Built on by (most cited first, 6 of 6)

- [Adaptive Reward Design for Reinforcement Learning](https://mlsanity.com/paper/2412.10917/) (2024)
- [Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Exploration](https://mlsanity.com/paper/2410.12136/) (2024)
- [Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications](https://mlsanity.com/paper/2311.17059/) (2023)
- [On Reducing Undesirable Behavior in Deep Reinforcement Learning Models](https://mlsanity.com/paper/2309.02869/) (2023)
- [Comp-LTL: Temporal Logic Planning via Zero-Shot Policy Composition](https://mlsanity.com/paper/2408.04215/) (2024)
- [Legislating World-Model-Based Planning with Legal Reasoning](https://mlsanity.com/paper/2609.15113/) (2026)

## Abstract (arXiv)

This paper explores continuous-time control synthesis for target-driven navigation to satisfy complex high-level tasks expressed as linear temporal logic (LTL). We propose a model-free framework using deep reinforcement learning (DRL) where the underlying dynamic system is unknown (an opaque box). Unlike prior work, this paper considers scenarios where the given LTL specification might be infeasible and therefore cannot be accomplished globally. Instead of modifying the given LTL formula, we provide a general DRL-based approach to satisfy it with minimal violation. To do this, we transform a previously multi-objective DRL problem, which requires simultaneous automata satisfaction and minimum violation cost, into a single objective. By guiding the DRL agent with a sampling-based path planning algorithm for the potentially infeasible LTL task, the proposed approach mitigates the myopic tendencies of DRL, which are often an issue when learning general LTL tasks that can have long or infinite horizons. This is achieved by decomposing an infeasible LTL formula into several reach-avoid sub-tasks with shorter horizons, which can be trained in a modular DRL architecture. Furthermore, we overcome the challenge of the exploration process for DRL in complex and cluttered environments by using path planners to design rewards that are dense in the configuration space. The benefits of the presented approach are demonstrated through testing on various complex nonlinear systems and compared with state-of-the-art baselines. The Video demonstration can be found here:https://youtu.be/jBhx6Nv224E.