# Worst-Case Discovery and Runtime Protection for RL-Based Network Controllers

arXiv: https://arxiv.org/abs/2605.04373 · HTML: https://mlsanity.com/paper/2605.04373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 6 May 2026.

## Builds on (most cited first, 14 of 14)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Automated Curriculum Learning for Neural Networks](https://mlsanity.com/paper/1704.03003/) (2017)
- [Automatic Curriculum Generation for Learning Adaptation in Networking](https://mlsanity.com/paper/2202.05940/) (2022)
- [Testing of Deep Reinforcement Learning Agents with Surrogate Models](https://mlsanity.com/paper/2305.12751/) (2023)
- [Optimistic Critic Reconstruction and Constrained Fine-Tuning for General Offline-to-Online RL](https://mlsanity.com/paper/2412.18855/) (2024)
- [Robustifying ML-powered Network Classifiers with PANTS](https://mlsanity.com/paper/2409.04691/) (2024)
- [CaFA: Cost-aware, Feasible Attacks With Database Constraints Against Neural Tabular Classifiers](https://mlsanity.com/paper/2501.10013/) (2025)
- [Finding Adversarial Inputs for Heuristics using Multi-level Optimization](https://mlsanity.com/paper/2311.12779/) (2023)
- [Mowgli: Passively Learned Rate Control for Real-Time Video](https://mlsanity.com/paper/2410.03339/) (2024)

## Abstract (arXiv)

RL-based controllers achieve strong average-case performance in networking tasks such as congestion control and adaptive bitrate streaming. Yet their performance can degrade severely under network conditions where strong performance is still achievable. Identifying such conditions and quantifying the resulting performance gap is intractable by enumeration, while the sequential and closed-loop nature of RL controllers makes formal verification methods impractical. We present ReGuard, a framework that discovers worst-case scenarios for a given RL controller and protects it against them at inference time without retraining. Discovery is formulated as a bilevel regret-maximization problem, which yields a certified lower bound on the worst-case performance gap. The discovered trajectories are then analyzed as counterfactuals and compiled into lightweight logic rules that intervene only when a risky state is detected, leaving the controller's behavior unchanged otherwise. We evaluate ReGuard across three RL-based network controllers: Pensieve, Sage, and Park. ReGuard discovers scenarios in which the controller's performance is 43-64% worse than what is achievable. ReGuard not only discovers gaps 57% to 6× larger than those found by the strongest baselines but also shrinks them by 79-85% via lightweight rule-based protection while preserving nominal performance. ReGuard's protection extends beyond the scenarios it discovers, improving performance across a wider range of network conditions.