# Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study

arXiv: https://arxiv.org/abs/2602.14322 · HTML: https://mlsanity.com/paper/2602.14322/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 15 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 10 of 10)

- [Robust Online Monitoring of Signal Temporal Logic](https://mlsanity.com/paper/1506.08234/) (2015)
- [Conformal Prediction for STL Runtime Verification](https://mlsanity.com/paper/2211.01539/) (2022)
- [On Robustness Metrics for Learning STL Tasks](https://mlsanity.com/paper/2003.06041/) (2020)
- [Tractable Reinforcement Learning of Signal Temporal Logic Objectives](https://mlsanity.com/paper/2001.09467/) (2020)
- [Online Shielding for Reinforcement Learning](https://mlsanity.com/paper/2212.01861/) (2022)
- [Model-based Reinforcement Learning from Signal Temporal Logic Specifications](https://mlsanity.com/paper/2011.04950/) (2020)
- [Robust Conformal Prediction for STL Runtime Verification under Distribution Shift](https://mlsanity.com/paper/2311.09482/) (2023)
- [Statistical Verification of Autonomous Systems using Surrogate Models and Conformal Inference](https://mlsanity.com/paper/2004.00279/) (2020)
- [STL-Based Synthesis of Feedback Controllers Using Reinforcement Learning](https://mlsanity.com/paper/2212.01022/) (2022)
- [Stress Testing Control Loops in Cyber-Physical Systems](https://mlsanity.com/paper/2302.13913/) (2023)

## Built on by (most cited first, 1 of 1)

- [Adaptive Safety Filtering for Frozen ACC Policies via Conformal Residual Calibration](https://mlsanity.com/paper/2609.35415/) (2026)

## Abstract (arXiv)

We investigate how formal temporal logic specifications can enhance the safety and robustness of reinforcement learning (RL) control in aerospace applications. Using the open source AeroBench F-16 simulation benchmark, we train a Proximal Policy Optimization (PPO) agent to regulate engine throttle and track commanded airspeed. The control objective is encoded as a Signal Temporal Logic (STL) requirement to maintain airspeed within a prescribed band during the final seconds of each maneuver. To enforce this specification at run time, we introduce a conformal STL shield that filters the RL agent's actions using online conformal prediction. We compare three settings: (i) PPO baseline, (ii) PPO with a classical rule-based STL shield, and (iii) PPO with the proposed conformal shield, under both nominal conditions and a severe stress scenario involving aerodynamic model mismatch, actuator rate limits, measurement noise, and mid-episode setpoint jumps. Experiments show that the conformal shield preserves STL satisfaction while maintaining near baseline performance and providing stronger robustness guarantees than the classical shield. These results demonstrate that combining formal specification monitoring with data driven RL control can substantially improve the reliability of autonomous flight control in challenging environments.