Worrisome Properties of Neural Network Controllers and Their Symbolic Representations
Not on the map. Why?
Abstract (arXiv)
We raise concerns about controllers' robustness in simple reinforcement learning benchmark problems. We focus on neural network controllers and their low neuron and symbolic abstractions. A typical controller reaching high mean return values still generates an abundance of persistent low-return solutions, which is a highly undesirable property, easily exploitable by an adversary. We find that the simpler controllers admit more persistent bad solutions. We provide an algorithm for a systematic robustness study and prove existence of persistent solutions and, in some cases, periodic orbits, using a computer-assisted proof methodology.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 43 references that are in the map. First posted on arXiv 28 Jul 2023.
Builds on
19 of 43 references.
References · 19
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
- 2018DeepMind Control Suitecited by 956
- 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
- 2017Robust Adversarial Reinforcement Learningcited by 451
- 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 344
- 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 242
- 2018Programmatically Interpretable Reinforcement Learningcited by 193
- 2017Spurious Local Minima are Common in Two-Layer ReLU Neural Networkscited by 104
7 more
- 2019On Robustness of Neural Ordinary Differential Equationscited by 83
- 2018Toward Interpretable Deep Reinforcement Learning with Linear Model U-Treescited by 65
- 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
- 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 48
- 2017Compositional Falsification of Cyber-Physical Systems with Machine Learning Componentscited by 32
- 2018Falsification of Cyber-Physical Systems Using Deep Reinforcement Learningcited by 17
- 2020Falsification of Cyber-Physical Systems with Robustness-Guided Black-Box Checkingcited by 7
Built on by
Not cited yet.