# Learning to Remember Translation History with a Continuous Cache

arXiv: https://arxiv.org/abs/1711.09367 · HTML: https://mlsanity.com/paper/1711.09367/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2017.

## Built on by (most cited first, 30 of 39)

- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [Adapting Large Language Models for Document-Level Machine Translation](https://mlsanity.com/paper/2401.06468/) (2024)
- [Measuring and Increasing Context Usage in Context-Aware Machine Translation](https://mlsanity.com/paper/2105.03482/) (2021)
- [Fast Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2105.14528/) (2021)
- [GNN-LM: Language Modeling based on Global Contexts via GNN](https://mlsanity.com/paper/2110.08743/) (2021)
- [Escaping the sentence-level paradigm in machine translation](https://mlsanity.com/paper/2304.12959/) (2023)
- [Do Context-Aware Translation Models Pay the Right Attention?](https://mlsanity.com/paper/2105.06977/) (2021)
- [Modeling Bilingual Conversational Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2107.11164/) (2021)
- [Towards Robust k-Nearest-Neighbor Machine Translation](https://mlsanity.com/paper/2210.08808/) (2022)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Diverse Pretrained Context Encodings Improve Document Translation](https://mlsanity.com/paper/2106.03717/) (2021)
- [Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Models](https://mlsanity.com/paper/2103.17151/) (2021)
- [Towards Making the Most of Dialogue Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2109.00668/) (2021)
- [Towards User-Driven Neural Machine Translation](https://mlsanity.com/paper/2106.06200/) (2021)
- [Investigating Length Issues in Document-level Machine Translation](https://mlsanity.com/paper/2412.17592/) (2024)
- [Integrating Vectorized Lexical Constraints for Neural Machine Translation](https://mlsanity.com/paper/2203.12210/) (2022)
- [Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level](https://mlsanity.com/paper/2308.13506/) (2023)
- [Non-Parametric Online Learning from Human Feedback for Neural Machine Translation](https://mlsanity.com/paper/2109.11136/) (2021)
- [Faster Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2112.08152/) (2021)
- [Improving Long Context Document-Level Machine Translation](https://mlsanity.com/paper/2306.05183/) (2023)
- [Scheduled Multi-task Learning for Neural Chat Translation](https://mlsanity.com/paper/2205.03766/) (2022)
- [Focused Concatenation for Context-Aware Neural Machine Translation](https://mlsanity.com/paper/2210.13388/) (2022)
- [Rethinking Nearest Neighbors for Visual Classification](https://mlsanity.com/paper/2112.08459/) (2021)
- [Nearest Neighbors Meet Deep Neural Networks for Point Cloud Analysis](https://mlsanity.com/paper/2303.00703/) (2023)
- [Challenges in Context-Aware Neural Machine Translation](https://mlsanity.com/paper/2305.13751/) (2023)
- [Modeling Context With Linear Attention for Scalable Document-Level Translation](https://mlsanity.com/paper/2210.08431/) (2022)
- [A Multi-task Multi-stage Transitional Training Framework for Neural Chat Translation](https://mlsanity.com/paper/2301.11749/) (2023)
- [Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach](https://mlsanity.com/paper/2409.09009/) (2024)
- [Exploring Paracrawl for Document-level Neural Machine Translation](https://mlsanity.com/paper/2304.10216/) (2023)
- [Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context](https://mlsanity.com/paper/2311.15507/) (2023)

## Abstract (arXiv)

Existing neural machine translation (NMT) models generally translate sentences in isolation, missing the opportunity to take advantage of document-level information. In this work, we propose to augment NMT models with a very light-weight cache-like memory network, which stores recent hidden representations as translation history. The probability distribution over generated words is updated online depending on the translation history retrieved from the memory, endowing NMT models with the capability to dynamically adapt over time. Experiments on multiple domains with different topics and styles show the effectiveness of the proposed approach with negligible impact on the computational cost.