KnowMe-Bench: Benchmarking Person Understanding for Lifelong Digital Companions
Not on the map. Why?
Abstract (arXiv)
Existing long-horizon memory benchmarks mostly use multi-turn dialogues or synthetic user histories, which makes retrieval performance an imperfect proxy for person understanding. We present BenchName, a publicly releasable benchmark built from long-form autobiographical narratives, where actions, context, and inner thoughts provide dense evidence for inferring stable motivations and decision principles. BenchName~reconstructs each narrative into a flashback-aware, time-anchored stream and evaluates models with evidence-linked questions spanning factual recall, subjective state attribution, and principle-level reasoning. Across diverse narrative sources, retrieval-augmented systems mainly improve factual accuracy, while errors persist on temporally grounded explanations and higher-level inferences, highlighting the need for memory mechanisms beyond retrieval. Our data is in hrefKnowMeBenchhttps://github.com/QuantaAlpha/KnowMeBench.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
16 of 29 references.
References · 16
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
- 2025Memory in the Age of AI Agentscited by 285
- 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
- 2024Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Taskscited by 102
- 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
- 2025MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agentscited by 82
- 2024VisionArena: 230K Real World User-VLM Conversations with Preference Labelscited by 22
4 more
- 2024Persona-L has Entered the Chat: Leveraging LLM and Ability-based Framework for Personas of People with Complex Needscited by 11
- 2025OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarioscited by 9
- 2024The influence of persona and conversational task on social interactions with a LLM-controlled embodied conversational agentcited by 7
- 2024Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevancecited by 7
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environmentscited by 13
- 2026LMEB: Long-horizon Memory Embedding Benchmarkcited by 6
- 2026DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settingscited by 3
- 2026PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agentscited by 2
- 2026HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?cited by 1
- 2026FlowTime: Towards Continuous Generative Watch Time Prediction via Flow-based Personalized Priorscited by 0
- 2026BehaviorBench: Modeling Real-World User Decisions from Behavioral Tracescited by 0
- 2026Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Datacited by 0
- 2026Know It, Act on It: Investigating Memory Utilization in LLM Personalizationcited by 0
- 2026Know It, Act on It: Investigating Memory Utilization in LLM Personalization31 Jul 2026
- 2026Setoka: A Benchmark for Hierarchical User Understanding in Personalized Agents over Heterogeneous Data29 Jul 2026
- 2026DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settings22 Jun 2026
- 2026PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agents3 Jun 2026
- 2026BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces1 Jun 2026
- 2026FlowTime: Towards Continuous Generative Watch Time Prediction via Flow-based Personalized Priors31 May 2026
- 2026HEART-Bench: Do LLM Agents Exhibit Human-like Psychology?28 May 2026
- 2026PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments24 Mar 2026
- 2026LMEB: Long-horizon Memory Embedding Benchmark13 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
- 2026RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations
- 2026IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscences
- 2026To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives
- 2026ES-MemEval: Benchmarking Conversational Agents on Personalized Long-Term Emotional Support
- 2026ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents
- 2026According to Me: Long-Term Personalized Referential Memory QA
- 2026PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversations