# PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answering

arXiv: https://arxiv.org/abs/2402.16288 · HTML: https://mlsanity.com/paper/2402.16288/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition](https://mlsanity.com/paper/2309.15112/) (2023)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [Open Question Answering over Tables and Text](https://mlsanity.com/paper/2010.10439/) (2020)
- [ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory](https://mlsanity.com/paper/2306.03901/) (2023)
- [Long Time No See! Open-Domain Conversation with Long-Term Persona Memory](https://mlsanity.com/paper/2203.05797/) (2022)
- [Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations](https://mlsanity.com/paper/2308.11995/) (2023)
- [Personalized Dialogue Generation with Diversified Traits](https://mlsanity.com/paper/1901.09672/) (2019)
- [Prompted LLMs as Chatbot Modules for Long Open-domain Conversation](https://mlsanity.com/paper/2305.04533/) (2023)
- [LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordination](https://mlsanity.com/paper/2309.11696/) (2023)
- [HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data](https://mlsanity.com/paper/2204.13243/) (2022)
- [A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models](https://mlsanity.com/paper/2311.16789/) (2023)
- [GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation](https://mlsanity.com/paper/2302.14401/) (2023)
- [UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning](https://mlsanity.com/paper/2306.10543/) (2023)

## Built on by (most cited first, 30 of 34)

- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents](https://mlsanity.com/paper/2506.21605/) (2025)
- [A Survey of Personalized Large Language Models: Progress and Future Directions](https://mlsanity.com/paper/2502.11528/) (2025)
- [MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems](https://mlsanity.com/paper/2510.17281/) (2025)
- [Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics](https://mlsanity.com/paper/2505.00675/) (2025)
- [Graph-based Agent Memory: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2602.05665/) (2026)
- [LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues](https://mlsanity.com/paper/2605.12493/) (2026)
- [Toward Multi-Session Personalized Conversation: A Large-Scale Dataset and Hierarchical Tree Framework for Implicit Reasoning](https://mlsanity.com/paper/2503.07018/) (2025)
- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information](https://mlsanity.com/paper/2508.13250/) (2025)
- [According to Me: Long-Term Personalized Referential Memory QA](https://mlsanity.com/paper/2603.01990/) (2026)
- [PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments](https://mlsanity.com/paper/2603.23231/) (2026)
- [Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey](https://mlsanity.com/paper/2506.11102/) (2025)
- [Convomem Benchmark: Why Your First 150 Conversations Don't Need RAG](https://mlsanity.com/paper/2511.10523/) (2025)
- [EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents](https://mlsanity.com/paper/2601.16690/) (2026)
- [Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning](https://mlsanity.com/paper/2602.18493/) (2026)
- [SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://mlsanity.com/paper/2410.20682/) (2024)
- [LMEB: Long-horizon Memory Embedding Benchmark](https://mlsanity.com/paper/2603.12572/) (2026)
- [MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models](https://mlsanity.com/paper/2605.14906/) (2026)
- [A Multi-Task Role-Playing Agent Capable of Imitating Character Linguistic Styles](https://mlsanity.com/paper/2411.02457/) (2024)
- [EvolMem: A Cognitive-Driven Benchmark for Multi-Session Dialogue Memory](https://mlsanity.com/paper/2601.03543/) (2026)
- [MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoning](https://mlsanity.com/paper/2603.03379/) (2026)
- [MemoryRewardBench: Benchmarking Reward Models for Long-Term Memory Management in Large Language Models](https://mlsanity.com/paper/2601.11969/) (2026)
- [VehicleMemBench: An Executable Benchmark for Multi-User Long-Term Memory in In-Vehicle Agents](https://mlsanity.com/paper/2603.23840/) (2026)
- [Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving LLM Agents](https://mlsanity.com/paper/2606.09483/) (2026)
- [MemoCue: Empowering LLM-Based Agents for Human Memory Recall via Strategy-Guided Querying](https://mlsanity.com/paper/2507.23633/) (2025)
- [Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory](https://mlsanity.com/paper/2601.00821/) (2025)
- [HiNS: Hierarchical Negative Sampling for More Comprehensive Memory Retrieval Embedding Model](https://mlsanity.com/paper/2601.14857/) (2026)

## Abstract (arXiv)

Long-term memory plays a critical role in personal interaction, considering long-term memory can better leverage world knowledge, historical information, and preferences in dialogues. Our research introduces PerLTQA, an innovative QA dataset that combines semantic and episodic memories, including world knowledge, profiles, social relationships, events, and dialogues. This dataset is collected to investigate the use of personalized memories, focusing on social interactions and events in the QA task. PerLTQA features two types of memory and a comprehensive benchmark of 8,593 questions for 30 characters, facilitating the exploration and application of personalized memories in Large Language Models (LLMs). Based on PerLTQA, we propose a novel framework for memory integration and generation, consisting of three main components: Memory Classification, Memory Retrieval, and Memory Synthesis. We evaluate this framework using five LLMs and three retrievers. Experimental results demonstrate that BERT-based classification models significantly outperform LLMs such as ChatGLM3 and ChatGPT in the memory classification task. Furthermore, our study highlights the importance of effective memory integration in the QA task.