# Model-based Reinforcement Learning from Signal Temporal Logic Specifications

arXiv: https://arxiv.org/abs/2011.04950 · HTML: https://mlsanity.com/paper/2011.04950/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2020.

## Built on by (most cited first, 18 of 18)

- [DeepSTL -- From English Requirements to Signal Temporal Logic](https://mlsanity.com/paper/2109.10294/) (2021)
- [Signal Temporal Logic Neural Predictive Control](https://mlsanity.com/paper/2309.05131/) (2023)
- [Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning](https://mlsanity.com/paper/2402.17217/) (2024)
- [Safe Deep Reinforcement Learning by Verifying Task-Level Properties](https://mlsanity.com/paper/2302.10030/) (2023)
- [TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matching](https://mlsanity.com/paper/2505.00562/) (2025)
- [Follow The Rules: Online Signal Temporal Logic Tree Search for Guided Imitation Learning in Stochastic Domains](https://mlsanity.com/paper/2209.13737/) (2022)
- [Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks](https://mlsanity.com/paper/2501.13457/) (2025)
- [TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks](https://mlsanity.com/paper/2510.00225/) (2025)
- [Logically Constrained Robotics Transformers for Enhanced Perception-Action Planning](https://mlsanity.com/paper/2408.05336/) (2024)
- [Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications](https://mlsanity.com/paper/2607.00442/) (2026)
- [Deep Reinforcement Learning Based Networked Control with Network Delays for Signal Temporal Logic Specifications](https://mlsanity.com/paper/2108.01317/) (2021)
- [Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study](https://mlsanity.com/paper/2602.14322/) (2026)
- [hint2: Hierarchical World Models for Inference-Time Temporal Logic Guidance](https://mlsanity.com/paper/2608.13678/) (2026)
- [Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout](https://mlsanity.com/paper/2403.15826/) (2024)
- [COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Management](https://mlsanity.com/paper/2603.02396/) (2026)
- [On Tackling Complex Tasks with Reward Machines and Signal Temporal Logics](https://mlsanity.com/paper/2604.14440/) (2026)
- [Robustness of Robotic Manipulation: Foundations and Frontiers](https://mlsanity.com/paper/2606.31494/) (2026)
- [Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning](https://mlsanity.com/paper/2609.37519/) (2026)

## Abstract (arXiv)

Techniques based on Reinforcement Learning (RL) are increasingly being used to design control policies for robotic systems. RL fundamentally relies on state-based reward functions to encode desired behavior of the robot and bad reward functions are prone to exploitation by the learning agent, leading to behavior that is undesirable in the best case and critically dangerous in the worst. On the other hand, designing good reward functions for complex tasks is a challenging problem. In this paper, we propose expressing desired high-level robot behavior using a formal specification language known as Signal Temporal Logic (STL) as an alternative to reward/cost functions. We use STL specifications in conjunction with model-based learning to design model predictive controllers that try to optimize the satisfaction of the STL specification over a finite time horizon. The proposed algorithm is empirically evaluated on simulations of robotic system such as a pick-and-place robotic arm, and adaptive cruise control for autonomous vehicles.