# Incorporating Copying Mechanism in Sequence-to-Sequence Learning

arXiv: https://arxiv.org/abs/1603.06393 · HTML: https://mlsanity.com/paper/1603.06393/

## Key facts

- Cited by 218 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2016.

## Built on by (most cited first, 30 of 218)

- [Retrieval Head Mechanistically Explains Long-Context Factuality](https://mlsanity.com/paper/2404.15574/) (2024)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation](https://mlsanity.com/paper/2305.09515/) (2023)
- [Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise](https://mlsanity.com/paper/2212.11685/) (2022)
- [Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization](https://mlsanity.com/paper/2110.07356/) (2021)
- [DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents](https://mlsanity.com/paper/2101.11796/) (2021)
- [Neural Machine Translation with Monolingual Translation Memory](https://mlsanity.com/paper/2105.11269/) (2021)
- [Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2211.07634/) (2022)
- [Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments](https://mlsanity.com/paper/2302.11649/) (2023)
- [CoditT5: Pretraining for Source Code and Natural Language Editing](https://mlsanity.com/paper/2208.05446/) (2022)
- [Temporal Knowledge Graph Reasoning with Historical Contrastive Learning](https://mlsanity.com/paper/2211.10904/) (2022)
- [One2Set: Generating Diverse Keyphrases as a Set](https://mlsanity.com/paper/2105.11134/) (2021)
- [The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction](https://mlsanity.com/paper/2104.06344/) (2021)
- [One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles](https://mlsanity.com/paper/2108.09355/) (2021)
- [Paradigm Shift in Natural Language Processing](https://mlsanity.com/paper/2109.12575/) (2021)
- [Neural Transfer Learning for Repairing Security Vulnerabilities in C Code](https://mlsanity.com/paper/2104.08308/) (2021)
- [Data-to-text Generation with Macro Planning](https://mlsanity.com/paper/2102.02723/) (2021)
- [Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing](https://mlsanity.com/paper/2103.02227/) (2021)
- [Opportunities and Challenges in Neural Dialog Tutoring](https://mlsanity.com/paper/2301.09919/) (2023)
- [Code Structure Guided Transformer for Source Code Summarization](https://mlsanity.com/paper/2104.09340/) (2021)
- [Semi-Supervised Variational Reasoning for Medical Dialogue Generation](https://mlsanity.com/paper/2105.06071/) (2021)
- [Focus Attention: Promoting Faithfulness and Diversity in Summarization](https://mlsanity.com/paper/2105.11921/) (2021)
- [Sequence-to-Sequence Learning with Latent Neural Grammars](https://mlsanity.com/paper/2109.01135/) (2021)
- [Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement](https://mlsanity.com/paper/2304.14391/) (2023)
- [Unobserved Local Structures Make Compositional Generalization Hard](https://mlsanity.com/paper/2201.05899/) (2022)
- [A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generation](https://mlsanity.com/paper/2203.15108/) (2022)
- [BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence](https://mlsanity.com/paper/2402.12174/) (2024)
- [SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising](https://mlsanity.com/paper/2105.07911/) (2021)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [A Survey on Neural Open Information Extraction: Current Status and Future Directions](https://mlsanity.com/paper/2205.11725/) (2022)

## Abstract (arXiv)

We address an important problem in sequence-to-sequence (Seq2Seq) learning referred to as copying, in which certain segments in the input sequence are selectively replicated in the output sequence. A similar phenomenon is observable in human language communication. For example, humans tend to repeat entity names or even long phrases in conversation. The challenge with regard to copying in Seq2Seq is that new machinery is needed to decide when to perform the operation. In this paper, we incorporate copying into neural network-based Seq2Seq learning and propose a new model called CopyNet with encoder-decoder structure. CopyNet can nicely integrate the regular way of word generation in the decoder with the new copying mechanism which can choose sub-sequences in the input sequence and put them at proper places in the output sequence. Our empirical study on both synthetic data sets and real world data sets demonstrates the efficacy of CopyNet. For example, CopyNet can outperform regular RNN-based model with remarkable margins on text summarization tasks.