# Implementing Neural Turing Machines

arXiv: https://arxiv.org/abs/1807.08518 · HTML: https://mlsanity.com/paper/1807.08518/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2018.

## Built on by (most cited first, 4 of 4)

- [Rationalizing Predictions by Adversarial Information Calibration](https://mlsanity.com/paper/2301.06009/) (2023)
- [Algorithmic Language Models with Neurally Compiled Libraries](https://mlsanity.com/paper/2407.04899/) (2024)
- [State-Space Constraints Can Improve the Generalisation of the Differentiable Neural Computer to Input Sequences With Unseen Length](https://mlsanity.com/paper/2110.09138/) (2021)
- [Large Language Models and Algorithm Execution: Application to an Arithmetic Function](https://mlsanity.com/paper/2601.07898/) (2026)

## Abstract (arXiv)

Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of NTMs exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a NTM. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.