Data to 14 Oct 2026

Lift Yourself Up: Retrieval-augmented Text Generation with Self Memory

arXiv 2305.02437 ↗First posted 3 May 2023cs.CL cs.AIcited by 103

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks. Since better memory would typically prompt better generation~(we define this as primal problem). The traditional approach for memory retrieval involves selecting memory that exhibits the highest similarity to the input. However, this method is constrained by the quality of the fixed corpus from which memory is retrieved. In this paper, by exploring the duality of the primal problem: better generation also prompts better memory, we propose a novel framework, selfmem, which addresses this limitation by iteratively employing a retrieval-augmented generator to create an unbounded memory pool and using a memory selector to choose one output as memory for the subsequent generation round. This enables the model to leverage its own output, referred to as self-memory, for improved generation. We evaluate the effectiveness of selfmem on three distinct text generation tasks: neural machine translation, abstractive text summarization, and dialogue generation, under two generation paradigms: fine-tuned small model and few-shot LLM. Our approach achieves state-of-the-art results in four directions in JRC-Acquis, XSum (50.3 ROUGE-1), and BigPatent (62.9 ROUGE-1), demonstrating the potential of self-memory in enhancing retrieval-augmented generation models. Furthermore, we conduct thorough analyses of each component in the selfmem framework to identify bottlenecks and provide insights for future research.

Key facts. Cited by 103 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 80 of its 118 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

80 of 118 references.

Closest ancestors · 1

Same direction, newest first

2022A Survey on Retrieval-Augmented Text Generation2 Feb

Other references · 79

2017Attention Is All You Needcited by 60,361
2020Language Models are Few-Shot Learnerscited by 32,419
2023GPT-4 Technical Reportcited by 18,396
2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
2014Sequence to Sequence Learning with Neural Networkscited by 2,986
2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918

38 more

2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
2020Big Bird: Transformers for Longer Sequencescited by 1,684
2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
2019Text Summarization with Pretrained Encoderscited by 556
2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
2021Few-shot Learning with Multilingual Language Modelscited by 278
2020Nearest Neighbor Machine Translationcited by 209
2015A Neural Conversational Modelcited by 196
2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
2022In-context Examples Selection for Machine Translationcited by 172
2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 156
2021Retrieval Augmented Code Generation and Summarizationcited by 149
2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 132
2022Retrieval-Augmented Multimodal Language Modelingcited by 125
2015Ask Me Anything: Dynamic Memory Networks for Natural Language Processingcited by 117
2020If beam search is the answer, what was the question?cited by 116
2022Training Language Models with Memory Augmentationcited by 113
2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
2020Residual Energy-Based Models for Text Generationcited by 95
2019On Extractive and Abstractive Neural Document Summarization with Transformer Language Modelscited by 83
2021Adaptive Semiparametric Language Modelscited by 83
2016Improving Neural Language Models with a Continuous Cachecited by 82
2018Retrieve and Refine: Improved Sequence Generation Models For Dialoguecited by 82
2020PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learningcited by 79

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 52; 2025: 35; 2026: 13.

103 papers.

Sort:

By area:

2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 212
2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 125
2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 88
2026Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Modelscited by 86
2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
2024From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMscited by 79

38 more

2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 71
2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 66
2025PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Pathscited by 61
2024FinTextQA: A Dataset for Long-form Financial Question Answeringcited by 54
2024Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Datacited by 52
2025Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Surveycited by 51
2024CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Modelscited by 50
2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 50
2024When Large Language Models Meet Vector Databases: A Surveycited by 39
2024M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitionscited by 38
2024MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memorycited by 35
2024CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrievalcited by 34
2025PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequencecited by 33
2024JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capabilitycited by 32
2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 31
2024Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrievalcited by 29
2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 27
2025NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodescited by 26
2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
2025GraphRAG under Firecited by 24
2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoorscited by 23
2024On Mitigating Code LLM Hallucinations with API Documentationcited by 21
2024Mixture of In-Context Experts Enhance LLMs' Long Context Awarenesscited by 17
2024SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Modelscited by 16
2024Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoningcited by 14
2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answeringcited by 14
2025A Cognitive Writing Perspective for Constrained Long-Form Text Generationcited by 14
2024FIT-RAG: Black-Box RAG with Factual Information and Token Reductioncited by 13
2024Towards LifeSpan Cognitive Systemscited by 13
2024EvoWiki: Evaluating LLMs on Evolving Knowledgecited by 13
2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13
2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
2025SRAG: Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Wikipedia Graphcited by 11
2025Personalize Before Retrieve: LLM-based Personalized Query Expansion for User-Centric Retrievalcited by 11
2024ARL2: Aligning Retrievers for Black-box Large Language Models via Self-guided Adaptive Relevance Labelingcited by 10
2025Privacy-Aware RAG: Secure and Isolated Knowledge Retrievalcited by 10
2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9

50 most cited of 103 All 103 (JSON)

Similar, not cited

Close in wording, no citation either way