# "Shall We Dig Deeper?": Designing and Evaluating Strategies for LLM Agents to Advance Knowledge Co-Construction in Asynchronous Online Discussions

arXiv: https://arxiv.org/abs/2509.23327 · HTML: https://mlsanity.com/paper/2509.23327/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 166 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making](https://mlsanity.com/paper/2301.05809/) (2023)
- [How should my chatbot interact? A survey on human-chatbot interaction design](https://mlsanity.com/paper/1904.02743/) (2019)
- [RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions](https://mlsanity.com/paper/2302.09466/) (2023)
- [Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making](https://mlsanity.com/paper/2403.16812/) (2024)
- ["Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2403.09552/) (2024)
- [DiscipLink: Unfolding Interdisciplinary Information Seeking Process via Human-AI Co-Exploration](https://mlsanity.com/paper/2408.00447/) (2024)
- [The Effects of System Initiative during Conversational Collaborative Search](https://mlsanity.com/paper/2202.09728/) (2022)
- [DBox: Scaffolding Algorithmic Programming Learning through Learner-LLM Co-Decomposition](https://mlsanity.com/paper/2502.19133/) (2025)
- [Engaged and Affective Virtual Agents: Their Impact on Social Presence, Trustworthiness, and Decision-Making in the Group Discussion](https://mlsanity.com/paper/2308.10385/) (2023)
- [Social-RAG: Retrieving from Group Interactions to Socially Ground AI Generation](https://mlsanity.com/paper/2411.02353/) (2024)
- [CoKnowledge: Supporting Assimilation of Time-synced Collective Knowledge in Online Science Videos](https://mlsanity.com/paper/2502.03767/) (2025)
- [Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences](https://mlsanity.com/paper/2507.08440/) (2025)
- [Engage Wider Audience or Facilitate Quality Answers? a Mixed-methods Analysis of Questioning Strategies for Research Sensemaking on a Community Q&A Site](https://mlsanity.com/paper/2311.10975/) (2023)
- [When Gamification Spoils Your Learning: A Qualitative Case Study of Gamification Misuse in a Language-Learning App](https://mlsanity.com/paper/2203.16175/) (2022)
- [Sharing Frissons among Online Video Viewers: Exploring the Design of Affective Communication for Aesthetic Chills](https://mlsanity.com/paper/2403.01090/) (2024)

## Built on by (most cited first, 1 of 1)

- [Collaborative Disagreement Resolution for Scalable Oversight](https://mlsanity.com/paper/2607.01251/) (2026)

## Abstract (arXiv)

Asynchronous online discussions enable diverse participants to co-construct knowledge beyond individual contributions. This process ideally evolves through sequential phases, from superficial information exchange to deeper synthesis. However, many discussions stagnate in the early stages. Existing AI interventions typically target isolated phases, lacking mechanisms to progressively advance knowledge co-construction, and the impacts of different intervention styles in this context remain unclear and warrant investigation. To address these gaps, we conducted a design workshop to explore AI intervention strategies (task-oriented and/or relationship-oriented) throughout the knowledge co-construction process, and implemented them in an LLM-powered agent capable of facilitating progression while consolidating foundations at each phase. A within-subject study (N=60) involving five consecutive asynchronous discussions showed that the agent consistently promoted deeper knowledge progression, with different styles exerting distinct effects on both content and experience. These findings provide actionable guidance for designing adaptive AI agents that sustain more constructive online discussions.