mlsanityData to 14 Oct 2026

When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being

arXiv 2512.08937 ↗First posted 24 Oct 2025cs.HC cs.AI cs.CYcited by 3

Not on the map. Why?

Abstract (arXiv)

Seeking advice is a core human behavior that the internet has reinvented twice: first through forums and Q&A communities that crowdsource public guidance, and now through large language models (LLMs). Yet the quality of this LLM advice for everyday well-being scenarios remains unclear. How does it compare, not only against human comments, but against the wisdom of the online crowd? We ran two studies (N=210) in which experts compared top-voted Reddit advice with LLM-generated advice. LLMs ranked significantly higher overall and on effectiveness, warmth, and willingness to seek advice again. GPT-4o beat GPT-5 on all metrics except sycophancy, suggesting that benchmark gains need not improve advice-giving. In Study-2, we examined how human and algorithmic advice could be combined, and found that human advice can be unobtrusively polished to compete with AI-generated comments. We conclude with design implications for advice-giving agents and ecosystems blending AI, crowd input, and expert oversight.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 19 of its 76 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

19 of 76 references.

References · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  7. 2024Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Taskscited by 106
  8. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 101
  9. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  10. 2025Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providerscited by 69
  11. 2022Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Supportcited by 68
  12. 2025Investigating Affective Use and Emotional Well-being on ChatGPTcited by 38
7 more
  1. 2024"Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Makingcited by 26
  2. 2024From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agentscited by 25
  3. 2023Computers as Bad Social Actors: Dark Patterns and Anti-Patterns in Interfaces that Act Sociallycited by 20
  4. 2025A Taxonomy of Linguistic Expressions That Contribute To Anthropomorphism of Language Technologiescited by 14
  5. 2025Letters from Future Self: Augmenting the Letter-Exchange Exercise with LLM-based Agents to Enhance Young Adults' Career Explorationcited by 8
  6. 2024Large Language Model Agents for Improving Engagement with Behavior Change Interventions: Application to Digital Mindfulnesscited by 4
  7. 2025Communication styles and reader preferences of LLM- and human-authored COVID-19 information explanations: a case studycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systemscited by 1
  2. 2026LLMs as Oracles: Reliance on LLMs for Subjective Personal Questionscited by 0
  3. 2026Trustworthy FinAInce: Unpacking How AI-Mediated Financial Advice is Judgedcited by 0