# Follow The Rules: Online Signal Temporal Logic Tree Search for Guided Imitation Learning in Stochastic Domains

arXiv: https://arxiv.org/abs/2209.13737 · HTML: https://mlsanity.com/paper/2209.13737/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Builds on (most cited first, 14 of 14)

- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation](https://mlsanity.com/paper/2205.03195/) (2022)
- [Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods](https://mlsanity.com/paper/2008.00097/) (2020)
- [RTAMT: Online Robustness Monitors from STL](https://mlsanity.com/paper/2005.11827/) (2020)
- [Recurrent Neural Network Controllers for Signal Temporal Logic Specifications Subject to Safety Constraints](https://mlsanity.com/paper/2009.11468/) (2020)
- [Model-based Reinforcement Learning from Signal Temporal Logic Specifications](https://mlsanity.com/paper/2011.04950/) (2020)
- [Learning from Demonstrations using Signal Temporal Logic](https://mlsanity.com/paper/2102.07730/) (2021)
- [Elaborating on Learned Demonstrations with Temporal Logic Specifications](https://mlsanity.com/paper/2002.00784/) (2020)
- [Semi-Supervised Trajectory-Feedback Controller Synthesis for Signal Temporal Logic Specifications](https://mlsanity.com/paper/2202.01997/) (2022)
- [Model-Free Reinforcement Learning for Optimal Control of MarkovDecision Processes Under Signal Temporal Logic Specifications](https://mlsanity.com/paper/2109.13377/) (2021)
- [Model Predictive Control for Signal Temporal Logic Specification](https://mlsanity.com/paper/1703.09563/) (2017)
- [STL2vec: Signal Temporal Logic Embeddings for Control Synthesis With Recurrent Neural Networks](https://mlsanity.com/paper/2109.04636/) (2021)
- [Temporal Logic Guided Motion Primitives for Complex Manipulation Tasks with User Preferences](https://mlsanity.com/paper/2202.04375/) (2022)

## Built on by (most cited first, 7 of 7)

- [Constrained Decoding for Safe Robot Navigation Foundation Models](https://mlsanity.com/paper/2509.01728/) (2025)
- [Logically Constrained Robotics Transformers for Enhanced Perception-Action Planning](https://mlsanity.com/paper/2408.05336/) (2024)
- [Pretrained Embeddings as a Behavior Specification Mechanism](https://mlsanity.com/paper/2503.02012/) (2025)
- [RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shifts](https://mlsanity.com/paper/2405.11139/) (2024)
- [Runtime Monitoring of Perception-Based Autonomous Systems via Embedding Temporal Logic](https://mlsanity.com/paper/2605.12651/) (2026)
- [Learned Tree Search for Long-Horizon Social Robot Navigation in Shared Airspace](https://mlsanity.com/paper/2304.01428/) (2023)
- [MCTS Based Dispatch of Autonomous Vehicles under Operational Constraints for Continuous Transportation](https://mlsanity.com/paper/2407.16200/) (2024)

## Abstract (arXiv)

Seamlessly integrating rules in Learning-from-Demonstrations (LfD) policies is a critical requirement to enable the real-world deployment of AI agents. Recently, Signal Temporal Logic (STL) has been shown to be an effective language for encoding rules as spatio-temporal constraints. This work uses Monte Carlo Tree Search (MCTS) as a means of integrating STL specification into a vanilla LfD policy to improve constraint satisfaction. We propose augmenting the MCTS heuristic with STL robustness values to bias the tree search towards branches with higher constraint satisfaction. While the domain-independent method can be applied to integrate STL rules online into any pre-trained LfD algorithm, we choose goal-conditioned Generative Adversarial Imitation Learning as the offline LfD policy. We apply the proposed method to the domain of planning trajectories for General Aviation aircraft around a non-towered airfield. Results using the simulator trained on real-world data showcase 60% improved performance over baseline LfD methods that do not use STL heuristics.