# A Primal-Dual Method for Training Recurrent Neural Networks Constrained by the Echo-State Property

arXiv: https://arxiv.org/abs/1311.6091 · HTML: https://mlsanity.com/paper/1311.6091/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2013.

## Built on by (most cited first, 1 of 1)

- [Differentiable Projection for Constrained Deep Learning](https://mlsanity.com/paper/2111.10785/) (2021)

## Abstract (arXiv)

We present an architecture of a recurrent neural network (RNN) with a fully-connected deep neural network (DNN) as its feature extractor. The RNN is equipped with both causal temporal prediction and non-causal look-ahead, via auto-regression (AR) and moving-average (MA), respectively. The focus of this paper is a primal-dual training method that formulates the learning of the RNN as a formal optimization problem with an inequality constraint that provides a sufficient condition for the stability of the network dynamics. Experimental results demonstrate the effectiveness of this new method, which achieves 18.86% phone recognition error on the TIMIT benchmark for the core test set. The result approaches the best result of 17.7%, which was obtained by using RNN with long short-term memory (LSTM). The results also show that the proposed primal-dual training method produces lower recognition errors than the popular RNN methods developed earlier based on the carefully tuned threshold parameter that heuristically prevents the gradient from exploding.