# Tractable Reinforcement Learning of Signal Temporal Logic Objectives

arXiv: https://arxiv.org/abs/2001.09467 · HTML: https://mlsanity.com/paper/2001.09467/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2020.

## Built on by (most cited first, 22 of 22)

- [Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic](https://mlsanity.com/paper/2102.12855/) (2021)
- [LTL-Constrained Policy Optimization with Cycle Experience Replay](https://mlsanity.com/paper/2404.11578/) (2024)
- [Model-Free Reinforcement Learning for Optimal Control of MarkovDecision Processes Under Signal Temporal Logic Specifications](https://mlsanity.com/paper/2109.13377/) (2021)
- [Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation](https://mlsanity.com/paper/2201.08504/) (2022)
- [Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning](https://mlsanity.com/paper/2212.03181/) (2022)
- [Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectives](https://mlsanity.com/paper/2202.02404/) (2022)
- [Learning Robust and Correct Controllers from Signal Temporal Logic Specifications Using BarrierNet](https://mlsanity.com/paper/2304.06160/) (2023)
- [Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks](https://mlsanity.com/paper/2501.13457/) (2025)
- [TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks](https://mlsanity.com/paper/2510.00225/) (2025)
- [Safe Model-based Control from Signal Temporal Logic Specifications Using Recurrent Neural Networks](https://mlsanity.com/paper/2103.15938/) (2021)
- [Deep Reinforcement Learning Based Networked Control with Network Delays for Signal Temporal Logic Specifications](https://mlsanity.com/paper/2108.01317/) (2021)
- [Neural Controller Synthesis for Signal Temporal Logic Specifications Using Encoder-Decoder Structured Networks](https://mlsanity.com/paper/2212.05200/) (2022)
- [Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study](https://mlsanity.com/paper/2602.14322/) (2026)
- [Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout](https://mlsanity.com/paper/2403.15826/) (2024)
- [Feasibility-aware Learning of Robust Temporal Logic Controllers using BarrierNet](https://mlsanity.com/paper/2512.06973/) (2025)
- [COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Management](https://mlsanity.com/paper/2603.02396/) (2026)
- [COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenance](https://mlsanity.com/paper/2603.07546/) (2026)
- [Specification-Aware Distribution Shaping for Robotics Foundation Models](https://mlsanity.com/paper/2603.17969/) (2026)
- [GraSP-STL: A Graph-Based Framework for Zero-Shot Signal Temporal Logic Planning via Offline Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2603.29533/) (2026)
- [Reward Machines for Signal Temporal Logic](https://mlsanity.com/paper/2608.13625/) (2026)
- [Tractable Reinforcement Learning for Full Class of Signal Temporal Logic Specifications Using Spatiotemporal Tube Reward](https://mlsanity.com/paper/2609.28396/) (2026)
- [Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning](https://mlsanity.com/paper/2609.37519/) (2026)

## Abstract (arXiv)

Signal temporal logic (STL) is an expressive language to specify time-bound real-world robotic tasks and safety specifications. Recently, there has been an interest in learning optimal policies to satisfy STL specifications via reinforcement learning (RL). Learning to satisfy STL specifications often needs a sufficient length of state history to compute reward and the next action. The need for history results in exponential state-space growth for the learning problem. Thus the learning problem becomes computationally intractable for most real-world applications. In this paper, we propose a compact means to capture state history in a new augmented state-space representation. An approximation to the objective (maximizing probability of satisfaction) is proposed and solved for in the new augmented state-space. We show the performance bound of the approximate solution and compare it with the solution of an existing technique via simulations.