# Echo State Neural Machine Translation

arXiv: https://arxiv.org/abs/2002.11847 · HTML: https://mlsanity.com/paper/2002.11847/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2020.

## Built on by (most cited first, 1 of 1)

- [Efficient and Private Federated Learning with Partially Trainable Networks](https://mlsanity.com/paper/2110.03450/) (2021)

## Abstract (arXiv)

We present neural machine translation (NMT) models inspired by echo state network (ESN), named Echo State NMT (ESNMT), in which the encoder and decoder layer weights are randomly generated then fixed throughout training. We show that even with this extremely simple model construction and training procedure, ESNMT can already reach 70-80% quality of fully trainable baselines. We examine how spectral radius of the reservoir, a key quantity that characterizes the model, determines the model behavior. Our findings indicate that randomized networks can work well even for complicated sequence-to-sequence prediction NLP tasks.