# REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation

arXiv: https://arxiv.org/abs/2502.13270 · HTML: https://mlsanity.com/paper/2502.13270/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2025.

## Built on by (most cited first, 30 of 35)

- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [REMem: Reasoning with Episodic Memory in Language Agent](https://mlsanity.com/paper/2602.13530/) (2026)
- [AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations](https://mlsanity.com/paper/2603.01966/) (2026)
- [Feel the Difference? A Comparative Analysis of Emotional Arcs in Real and LLM-Generated CBT Sessions](https://mlsanity.com/paper/2508.20764/) (2025)
- [EpiCache: Episodic KV Cache Management for Long-Term Conversation on Resource-Constrained Environments](https://mlsanity.com/paper/2509.17396/) (2025)
- [LMEB: Long-horizon Memory Embedding Benchmark](https://mlsanity.com/paper/2603.12572/) (2026)
- [PM-Bench: Evaluating Prospective Memory in LLM Agents](https://mlsanity.com/paper/2607.12385/) (2026)
- [AMA: Adaptive Memory via Multi-Agent Collaboration](https://mlsanity.com/paper/2601.20352/) (2026)
- [Diagnosing Retrieval vs. Utilization Bottlenecks in LLM Agent Memory](https://mlsanity.com/paper/2603.02473/) (2026)
- [Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams](https://mlsanity.com/paper/2603.07392/) (2026)
- [D-Mem: A Dual-Process Memory System for LLM Agents](https://mlsanity.com/paper/2603.18631/) (2026)
- [MemGym: a Long-Horizon Memory Environment for LLM Agents](https://mlsanity.com/paper/2605.20833/) (2026)
- [H-Mem: A Novel Memory Mechanism for Evolving and Retrieving Agent Memory via a Hybrid Structure](https://mlsanity.com/paper/2605.15701/) (2026)
- [Show or Tell? Modeling the evolution of request-making in Human-LLM conversations](https://mlsanity.com/paper/2508.01213/) (2025)
- [Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability](https://mlsanity.com/paper/2607.26637/) (2026)
- [Mitigating Over-Personalization in LLMs via Structured Memory](https://mlsanity.com/paper/2608.08300/) (2026)
- [MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare](https://mlsanity.com/paper/2605.11814/) (2026)
- [Tangram: Unlocking Non-Uniform KV Cache Compression for Efficient Multi-turn LLM Serving](https://mlsanity.com/paper/2606.06302/) (2026)
- [EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory](https://mlsanity.com/paper/2606.21649/) (2026)
- [Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KV](https://mlsanity.com/paper/2607.23693/) (2026)
- [Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline](https://mlsanity.com/paper/2606.04315/) (2026)
- [CoMemBench: Benchmarking Collaborative Memory Boundaries across Multi-Agent Workflow Topologies](https://mlsanity.com/paper/2609.32192/) (2026)
- [Which Questions Improve Learning the Most? Utility Estimation of Questions with LM-based Simulations](https://mlsanity.com/paper/2502.17383/) (2025)
- [On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputs](https://mlsanity.com/paper/2510.05864/) (2025)
- [Towards Continuous Sign Language Conversation from Isolated Signs](https://mlsanity.com/paper/2605.14705/) (2026)
- [Connecting the Dots: Benchmarking Reflective Memory in Long-Horizon Dialogue](https://mlsanity.com/paper/2606.01223/) (2026)
- [BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces](https://mlsanity.com/paper/2606.02798/) (2026)
- [Don't Scroll Back: Missing-Evidence Memory for Streaming Dialogue Summarization](https://mlsanity.com/paper/2608.09043/) (2026)
- [MemUse: Moving Memory Evaluation from Direct QA to Natural Integration in Long-Term Human-AI Conversation](https://mlsanity.com/paper/2608.24189/) (2026)

## Abstract (arXiv)

Long-term, open-domain dialogue capabilities are essential for chatbots aiming to recall past interactions and demonstrate emotional intelligence (EI). Yet, most existing research relies on synthetic, LLM-generated data, leaving open questions about real-world conversational patterns. To address this gap, we introduce REALTALK, a 21-day corpus of authentic messaging app dialogues, providing a direct benchmark against genuine human interactions. We first conduct a dataset analysis, focusing on EI attributes and persona consistency to understand the unique challenges posed by real-world dialogues. By comparing with LLM-generated conversations, we highlight key differences, including diverse emotional expressions and variations in persona stability that synthetic dialogues often fail to capture. Building on these insights, we introduce two benchmark tasks: (1) persona simulation where a model continues a conversation on behalf of a specific user given prior dialogue context; and (2) memory probing where a model answers targeted questions requiring long-term memory of past interactions. Our findings reveal that models struggle to simulate a user solely from dialogue history, while fine-tuning on specific user chats improves persona emulation. Additionally, existing models face significant challenges in recalling and leveraging long-term context within real-world conversations.