# Scenario-Assisted Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2202.04337 · HTML: https://mlsanity.com/paper/2202.04337/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 58 references that are in the map.
- First posted on arXiv 9 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [Mastering Complex Control in MOBA Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1912.09729/) (2019)
- [On a Formal Model of Safe and Scalable Self-driving Cars](https://mlsanity.com/paper/1708.06374/) (2017)
- [Online Multi-Target Tracking Using Recurrent Neural Networks](https://mlsanity.com/paper/1604.03635/) (2016)
- [Reward Shaping via Meta-Learning](https://mlsanity.com/paper/1901.09330/) (2019)
- [Guarded Deep Learning using Scenario-Based Modeling](https://mlsanity.com/paper/2006.03863/) (2020)
- [On-the-Fly Construction of Composite Events in Scenario-Based Modeling using Constraint Solvers](https://mlsanity.com/paper/1909.00408/) (2019)
- [Context-Oriented Behavioral Programming](https://mlsanity.com/paper/2005.02373/) (2020)
- [Towards Repairing Scenario-Based Models with Rich Events](https://mlsanity.com/paper/2101.03504/) (2021)

## Built on by (most cited first, 8 of 8)

- [Local vs. Global Interpretability: A Computational Complexity Perspective](https://mlsanity.com/paper/2406.02981/) (2024)
- [Formally Verifying Deep Reinforcement Learning Controllers with Lyapunov Barrier Certificates](https://mlsanity.com/paper/2405.14058/) (2024)
- [Verification-Guided Shielding for Deep Reinforcement Learning](https://mlsanity.com/paper/2406.06507/) (2024)
- [Constrained Reinforcement Learning for Robotics via Scenario-Based Programming](https://mlsanity.com/paper/2206.09603/) (2022)
- [Hard to Explain: On the Computational Hardness of In-Distribution Model Interpretation](https://mlsanity.com/paper/2408.03915/) (2024)
- [Towards Formal Verification of Deep Neural Networks for Object Detection](https://mlsanity.com/paper/2407.01295/) (2024)
- [Enhancing Deep Learning with Scenario-Based Override Rules: a Case Study](https://mlsanity.com/paper/2301.08114/) (2023)
- [Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning](https://mlsanity.com/paper/2405.20534/) (2024)

## Abstract (arXiv)

Deep reinforcement learning has proven remarkably useful in training agents from unstructured data. However, the opacity of the produced agents makes it difficult to ensure that they adhere to various requirements posed by human engineers. In this work-in-progress report, we propose a technique for enhancing the reinforcement learning training process (specifically, its reward calculation), in a way that allows human engineers to directly contribute their expert knowledge, making the agent under training more likely to comply with various relevant constraints. Moreover, our proposed approach allows formulating these constraints using advanced model engineering techniques, such as scenario-based modeling. This mix of black-box learning-based tools with classical modeling approaches could produce systems that are effective and efficient, but are also more transparent and maintainable. We evaluated our technique using a case-study from the domain of internet congestion control, obtaining promising results.