# Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes

arXiv: https://arxiv.org/abs/1607.00036 · HTML: https://mlsanity.com/paper/1607.00036/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2016.

## Built on by (most cited first, 8 of 8)

- [Recurrent Memory Transformer](https://mlsanity.com/paper/2207.06881/) (2022)
- [Scaling Transformer to 1M tokens and beyond with RMT](https://mlsanity.com/paper/2304.11062/) (2023)
- [Combining Modular Skills in Multitask Learning](https://mlsanity.com/paper/2202.13914/) (2022)
- [Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoning](https://mlsanity.com/paper/2106.03921/) (2021)
- [A survey on GANs for computer vision: Recent research, analysis and taxonomy](https://mlsanity.com/paper/2203.11242/) (2022)
- [Discrete Neural Algorithmic Reasoning](https://mlsanity.com/paper/2402.11628/) (2024)
- [Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture](https://mlsanity.com/paper/2310.03052/) (2023)
- [Complexity of Symbolic Representation in Working Memory of Transformer Correlates with the Complexity of a Task](https://mlsanity.com/paper/2406.14213/) (2024)

## Abstract (arXiv)

We extend neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing a trainable memory addressing scheme. This addressing scheme maintains for each memory cell two separate vectors, content and address vectors. This allows the D-NTM to learn a wide variety of location-based addressing strategies including both linear and nonlinear ones. We implement the D-NTM with both continuous, differentiable and discrete, non-differentiable read/write mechanisms. We investigate the mechanisms and effects of learning to read and write into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRUcontroller. The D-NTM is evaluated on a set of Facebook bAbI tasks and shown to outperform NTM and LSTM baselines. We have done extensive analysis of our model and different variations of NTM on bAbI task. We also provide further experimental results on sequential pMNIST, Stanford Natural Language Inference, associative recall and copy tasks.