mlsanityData to 14 Oct 2026

Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations

arXiv 2408.15232 ↗First posted 27 Aug 2024cs.CL cs.AI cs.IRcited by 43

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

While language model (LM)-powered chatbots and generative search engines excel at answering concrete queries, discovering information in the terrain of unknown unknowns remains challenging for users. To emulate the common educational scenario where children/students learn by listening to and participating in conversations of their parents/teachers, we create Collaborative STORM (Co-STORM). Unlike QA systems that require users to ask all the questions, Co-STORM lets users observe and occasionally steer the discourse among several LM agents. The agents ask questions on the user's behalf, allowing the user to discover unknown unknowns serendipitously. To facilitate user interaction, Co-STORM assists users in tracking the discourse by organizing the uncovered information into a dynamic mind map, ultimately generating a comprehensive report as takeaways. For automatic evaluation, we construct the WildSeek dataset by collecting real information-seeking records with user goals. Co-STORM outperforms baseline methods on both discourse trace and report quality. In a further human evaluation, 70% of participants prefer Co-STORM over a search engine, and 78% favor it over a RAG chatbot.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 31 of its 61 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

31 of 61 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration3 Oct
  2. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration11 Jul
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 28

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  7. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  10. 2018CoQA: A Conversational Question Answering Challengecited by 683
  11. 2023Enabling Large Language Models to Generate Text with Citationscited by 595
  12. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
16 more
  1. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  2. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  3. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  4. 2023Evaluating Verifiability in Generative Search Enginescited by 303
  5. 2020Open Question Answering over Tables and Textcited by 163
  6. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 154
  7. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  8. 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Datacited by 28
  9. 2024TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervisioncited by 25
  10. 2023Beyond Summarization: Designing AI Support for Real-World Expository Writing Taskscited by 20
  11. 2019User Intent Prediction in Information-seeking Conversationscited by 19
  12. 2019Interactive Machine Comprehension with Information Seeking Agentscited by 13
  13. 2022Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contextscited by 12
  14. 2024KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questionscited by 11
  15. 2021Open-Domain Question-Answering for COVID-19 and Other Emergent Domainscited by 7
  16. 2022Making Large Language Models Interactive: A Pioneer Study on Supporting Complex Information-Seeking Tasks with Implicit Constraintscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 26; 2026: 17.

43 papers.

Sort:

By area:

  1. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 238
  2. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 124
  3. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  4. 2025Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforcecited by 38
  5. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 29
  6. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  7. 2025RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discoverycited by 19
  8. 2025OmniThink: Expanding Knowledge Boundaries in Machine Writing through Thinkingcited by 18
  9. 2025Beyond Outlining: Heterogeneous Recursive Planning for Adaptive Long-form Writing with Language Modelscited by 16
  10. 2025TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applicationscited by 16
  11. 2025SciSage: A Multi-Agent Framework for High-Quality Scientific Survey Generationcited by 12
  12. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
31 more
  1. 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notescited by 8
  2. 2025XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaborationcited by 7
  3. 2026DREAM: Deep Research Evaluation with Agentic Metricscited by 7
  4. 2026DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytellingcited by 7
  5. 2025WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generationcited by 5
  6. 2025Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratingscited by 5
  7. 2025Interaction as Intelligence: Deep Research With Human-AI Partnershipcited by 5
  8. 2025Perspectra: Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideationcited by 4
  9. 2026Grounding Agent Memory in Contextual Intentcited by 4
  10. 2026Deep-Research Agents Can Be Poisoned via User-Generated Contentcited by 4
  11. 2025Test-time Corpus Feedback: From Retrieval to RAGcited by 3
  12. 2025HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Storescited by 3
  13. 2026Multi-Agent Debate Strategies: Survey, Taxonomy, and Challengescited by 3
  14. 2025LLM×MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resourcescited by 2
  15. 2026PROPER Agents: Proactivity Driven Personalized Agents for Advancing Knowledge Gap Navigationcited by 2
  16. 2026Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Userscited by 2
  17. 2026Self-Evolving Deep Research via Joint Generation and Evaluationcited by 2
  18. 2025DeFine: A Decomposed and Fine-Grained Annotated Dataset for Long-form Article Generationcited by 1
  19. 2025Goal2Story: A Multi-Agent Fleet based on Privately Enabled sLLMs for Impacting Mapping on Requirements Elicitationcited by 1
  20. 2025PAKTON: A Multi-Agent Framework for Question Answering in Long Legal Agreementscited by 1
  21. 2025DeepWriter: A Fact-Grounded Multimodal Writing Assistant Based On Offline Knowledge Basecited by 1
  22. 2026SoulSeek: Exploring the Use of Social Cues in LLM-based Information Seekingcited by 1
  23. 2026A Concise Agent is Less Expert: Revealing Side Effects of Using Style Features on Conversational Agentscited by 1
  24. 2025Writing Like the Best: Exemplar-Based Expository Text Generationcited by 0
  25. 2025ConvergeWriter: Data-Driven Bottom-Up Article Constructioncited by 0
  26. 2026DeALOG: Decentralized Multi-Agents Log-Mediated Reasoning Frameworkcited by 0
  27. 2026MEMENTO: Leveraging Web as a Learning Signal for Low-Data Domainscited by 0
  28. 2026Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentationcited by 0
  29. 2026Wyvern: An Agentic Framework for Generating Grounded Multimodal Reportscited by 0
  30. 2026CWF: A Collaborative Writing Framework for Personalized and Reliable Popular Science Writingcited by 0
  31. 2026Search Shapes Conclusions: Auditing Evidence Selection Bias in Deep Research Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025InfoQuest: Evaluating Multi-Turn Dialogue Agents for Open-Ended Conversations with Hidden Context
  2. 2023Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses
  3. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Tasks
  4. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents
  5. 2024Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks World
  6. 2026GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations
  7. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent
  8. 2025In Dialogue with Intelligence: Rethinking Large Language Models as Collective Knowledge