mlsanityData to 7 Oct 2026

MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation

arXiv 2308.08239 ↗First posted 16 Aug 2023cs.CLcited by 94

Not on the map. Why?

Cited by 50 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose MemoChat, a pipeline for refining instructions that enables large language models (LLMs) to effectively employ self-composed memos for maintaining consistent long-range open-domain conversations. We demonstrate a long-range open-domain conversation through iterative "memorization-retrieval-response" cycles. This requires us to carefully design tailored tuning instructions for each distinct stage. The instructions are reconstructed from a collection of public datasets to teach the LLMs to memorize and retrieve past dialogues with structured memos, leading to enhanced consistency when participating in future conversations. We invite experts to manually annotate a test set designed to evaluate the consistency of long-range conversations questions. Experiments on three testing scenarios involving both open-source and API-accessible chatbots at scale verify the efficacy of MemoChat, which outperforms strong baselines. Our codes, data and models are available here: https://github.com/LuJunru/MemoChat.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

54 of 68 references.

References · 54

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,362
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2020Reformer: The Efficient Transformercited by 1,658
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  12. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  13. 2022A Survey on In-context Learningcited by 717
  14. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  15. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  16. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  17. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  18. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  19. 2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 302
  20. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  21. 2021Internet-Augmented Dialogue Generationcited by 217
  22. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  23. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  24. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  25. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 181
  26. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  27. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  28. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 150
  29. 2023ChatDB: Augmenting LLMs with Databases as Their Symbolic Memorycited by 136
  30. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  31. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 107
  32. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  33. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  34. 2021TopiOCQA: Open-domain Conversational Question Answering with Topic Switchingcited by 95
  35. 2020doc2dial: A Goal-Oriented Document-Grounded Dialogue Datasetcited by 88
  36. 2023Aligning Large Language Models through Synthetic Feedbackcited by 75
  37. 2023Prompted LLMs as Chatbot Modules for Long Open-domain Conversationcited by 61
  38. 2020KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversationcited by 59

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 17; 2025: 39; 2026: 35.

94 papers.

Sort:

By area:

  1. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  2. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  3. 2025Memory in the Age of AI Agentscited by 285
  4. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  5. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  6. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  7. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 112
  8. 2025In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agentscited by 102
  9. 2025On Memory Construction and Retrieval for Personalized Conversational Agentscited by 101
  10. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  11. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  12. 2024Large Language Models Empowered Personalized Web Agentscited by 64
38 more
  1. 2023Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Modelscited by 62
  2. 2025MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systemscited by 56
  3. 2024Towards Lifelong Dialogue Agents via Timeline-based Memory Managementcited by 50
  4. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  5. 2025Mem-α: Learning Memory Construction via Reinforcement Learningcited by 49
  6. 2025PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistantscited by 44
  7. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  8. 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systemscited by 38
  9. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topicscited by 36
  10. 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
  11. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 35
  12. 2024LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Modelcited by 34
  13. 2024INMS: Memory Sharing for Large Language Model based Agentscited by 31
  14. 2025From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agentscited by 27
  15. 2025TReMu: Towards Neuro-Symbolic Temporal Reasoning for LLM-Agents with Memory in Multi-Session Dialoguescited by 25
  16. 2024MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistantscited by 24
  17. 2023A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Modelscited by 23
  18. 2024On the Way to LLM Personalization: Learning to Remember User Conversationscited by 21
  19. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  20. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  21. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  22. 2025MemGuide: Intent-Driven Memory Selection for Goal-Oriented Multi-Session LLM Agentscited by 16
  23. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  24. 2026Memory in the LLM Era: Modular Architectures and Strategies in a Unified Frameworkcited by 12
  25. 2024Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialoguecited by 11
  26. 2025Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interactioncited by 11
  27. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 11
  28. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  29. 2025HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistantscited by 9
  30. 2024Toward Conversational Agents with Context and Time Sensitive Long-term Memorycited by 8
  31. 2025MOOM: Maintenance, Organization and Optimization of Memory in Ultra-Long Role-Playing Dialoguescited by 8
  32. 2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
  33. 2026How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistantscited by 7
  34. 2024SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Scriptcited by 6
  35. 2026Semantic XPath: Structured Agentic Memory Access for Conversational AIcited by 6
  36. 2025ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratchcited by 5
  37. 2025A Multi-Memory Segment System for Generating High-Quality Long-Term Memory Content in Agentscited by 5
  38. 2026Self-Consolidation for Self-Evolving Agentscited by 5

50 most cited of 94 All 94 (JSON)