mlsanityData to 7 Oct 2026

CL-bench Life: Can Language Models Learn from Real-Life Context?

arXiv 2604.27043 ↗First posted 29 Apr 2026cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Today's AI assistants such as OpenClaw are designed to handle context effectively, making context learning an increasingly important capability for models. As these systems move beyond professional settings into everyday life, the nature of the contexts they must handle also shifts. Real-life contexts are often messy, fragmented, and deeply tied to personal and social experience, such as multi-party conversations, personal archives, and behavioral traces. Yet it remains unclear whether current frontier language models can reliably learn from such contexts and solve tasks grounded in them. To this end, we introduce CL-bench Life, a fully human-curated benchmark comprising 405 context-task pairs and 5,348 verification rubrics, covering common real-life scenarios. Solving tasks in CL-bench Life requires models to reason over complex, messy real-life contexts, calling for strong real-life context learning abilities that go far beyond those evaluated in existing benchmarks. We evaluate ten frontier LMs and find that real-life context learning remains highly challenging: even the best-performing model achieves only 19.3% task solving rate, while the average performance across models is only 13.8%. Models still struggle to reason over contexts such as messy group chat histories and fragmented behavioral records from everyday life. CL-bench Life provides a crucial testbed for advancing real-life context learning, and progress on it can enable more intelligent and reliable AI assistants in everyday life.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 57 of its 72 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

57 of 72 references.

References · 57

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  9. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  10. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  11. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  12. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
38 more
  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  2. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  3. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  6. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  7. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  9. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  10. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  11. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  12. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  13. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  14. 2023Cognitive Architectures for Language Agentscited by 422
  15. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  16. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  17. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
  18. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
  19. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  20. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  21. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  22. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
  23. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  24. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
  25. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 171
  26. 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
  27. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  28. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  29. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  30. 2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 89
  31. 2024LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAcited by 78
  32. 2025Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agentscited by 75
  33. 2023Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Taskscited by 73
  34. 2023BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Modelscited by 53
  35. 2026CL-bench: A Benchmark for Context Learningcited by 46
  36. 2025VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applicationscited by 45
  37. 2024VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Taskscited by 43
  38. 2025Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agentscited by 39

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Context-CoT: Enhancing Context Learning via High-Quality Reasoning Synthesiscited by 2
  2. 2026SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailingcited by 1
  3. 2026Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoningcited by 1
  4. 2026Agents in the Large: Perception-Centered Architecture for Persistent Agentscited by 1
  5. 2026EvolveScaler: Synthesizing Information-Evolution Contexts via Executable State Machines and Natural-Language Renderingcited by 0
  6. 2026Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Tracescited by 0
  7. 2026ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worldscited by 0
  8. 2026Learning to Learn from Context: Synthetic Training from Perturbed Public Documentscited by 0