# Robust Learning of Recurrent Neural Networks in Presence of Exogenous Noise

arXiv: https://arxiv.org/abs/2105.00996 · HTML: https://mlsanity.com/paper/2105.00996/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 13 of 13)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://mlsanity.com/paper/1801.10578/) (2018)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [Crafting Adversarial Input Sequences for Recurrent Neural Networks](https://mlsanity.com/paper/1604.08275/) (2016)
- [Gradient Descent Learns Linear Dynamical Systems](https://mlsanity.com/paper/1609.05191/) (2016)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [POPQORN: Quantifying Robustness of Recurrent Neural Networks](https://mlsanity.com/paper/1905.07387/) (2019)
- [Growing Linear Consensus Networks Endowed by Spectral Systemic Performance Measures](https://mlsanity.com/paper/1706.07832/) (2017)
- [Koopman Performance Analysis of Nonlinear Consensus Networks](https://mlsanity.com/paper/1807.04237/) (2018)
- [Explicit Characterization of Performance of a Class of Networked Linear Control Systems](https://mlsanity.com/paper/1908.01421/) (2019)
- [Distributed Map Classification using Local Observations](https://mlsanity.com/paper/2012.10480/) (2020)

## Built on by (most cited first, 3 of 3)

- [Robustness Analysis of Classification Using Recurrent Neural Networks with Perturbed Sequential Input](https://mlsanity.com/paper/2203.05403/) (2022)
- [A General Framework for Uncertainty Quantification via Neural SDE-RNN](https://mlsanity.com/paper/2306.01189/) (2023)
- [Symbolic Perception Risk in Autonomous Driving](https://mlsanity.com/paper/2303.09416/) (2023)

## Abstract (arXiv)

Recurrent Neural networks (RNN) have shown promising potential for learning dynamics of sequential data. However, artificial neural networks are known to exhibit poor robustness in presence of input noise, where the sequential architecture of RNNs exacerbates the problem. In this paper, we will use ideas from control and estimation theories to propose a tractable robustness analysis for RNN models that are subject to input noise. The variance of the output of the noisy system is adopted as a robustness measure to quantify the impact of noise on learning. It is shown that the robustness measure can be estimated efficiently using linearization techniques. Using these results, we proposed a learning method to enhance robustness of a RNN with respect to exogenous Gaussian noise with known statistics. Our extensive simulations on benchmark problems reveal that our proposed methodology significantly improves robustness of recurrent neural networks.