# A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks

arXiv: https://arxiv.org/abs/1709.09611 · HTML: https://mlsanity.com/paper/1709.09611/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2017.

## Built on by (most cited first, 20 of 20)

- [LTL2Action: Generalizing LTL Instructions for Multi-Task RL](https://mlsanity.com/paper/2102.06858/) (2021)
- [Reward Machines for Deep RL in Noisy and Uncertain Environments](https://mlsanity.com/paper/2406.00120/) (2024)
- [Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machines](https://mlsanity.com/paper/2211.10902/) (2022)
- [Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning](https://mlsanity.com/paper/2212.03181/) (2022)
- [Hierarchical Potential-based Reward Shaping from Task Specifications](https://mlsanity.com/paper/2110.02792/) (2021)
- [Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectives](https://mlsanity.com/paper/2202.02404/) (2022)
- [Possibility Before Utility: Learning And Using Hierarchical Affordances](https://mlsanity.com/paper/2203.12686/) (2022)
- [STL-Based Synthesis of Feedback Controllers Using Reinforcement Learning](https://mlsanity.com/paper/2212.01022/) (2022)
- [Learning Symbolic Representations for Reinforcement Learning of Non-Markovian Behavior](https://mlsanity.com/paper/2301.02952/) (2023)
- [Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Framework](https://mlsanity.com/paper/2103.12558/) (2021)
- [Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networks](https://mlsanity.com/paper/2103.15938/) (2021)
- [Automaton Constrained Q-Learning](https://mlsanity.com/paper/2510.05061/) (2025)
- [Deep Reinforcement Learning Based Networked Control with Network Delays for Signal Temporal Logic Specifications](https://mlsanity.com/paper/2108.01317/) (2021)
- [Neural Controller Synthesis for Signal Temporal Logic Specifications Using Encoder-Decoder Structured Networks](https://mlsanity.com/paper/2212.05200/) (2022)
- [Layered controller synthesis for dynamic multi-agent systems](https://mlsanity.com/paper/2307.06758/) (2023)
- [A Safe Preference Learning Approach for Personalization with Applications to Autonomous Vehicles](https://mlsanity.com/paper/2311.02099/) (2023)
- [Safe Policy Improvement in Constrained Markov Decision Processes](https://mlsanity.com/paper/2210.11259/) (2022)
- [Fulfilling Formal Specifications ASAP by Model-free Reinforcement Learning](https://mlsanity.com/paper/2304.12508/) (2023)
- [Control Synthesis of Cyber-Physical Systems for Real-Time Specifications through Causation-Guided Reinforcement Learning](https://mlsanity.com/paper/2510.07715/) (2025)
- [Live LTL Progress Tracking: Towards Task-Based Exploration](https://mlsanity.com/paper/2604.17106/) (2026)

## Abstract (arXiv)

Reward engineering is an important aspect of reinforcement learning. Whether or not the user's intentions can be correctly encapsulated in the reward function can significantly impact the learning outcome. Current methods rely on manually crafted reward functions that often require parameter tuning to obtain the desired behavior. This operation can be expensive when exploration requires systems to interact with the physical world. In this paper, we explore the use of temporal logic (TL) to specify tasks in reinforcement learning. TL formula can be translated to a real-valued function that measures its level of satisfaction against a trajectory. We take advantage of this function and propose temporal logic policy search (TLPS), a model-free learning technique that finds a policy that satisfies the TL specification. A set of simulated experiments are conducted to evaluate the proposed approach.