mlsanityData to 7 Oct 2026

Auditing Support Strategies in LLMs through Grounded Multi-Turn Social Simulation

arXiv 2604.17079 ↗First posted 18 Apr 2026cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

When users seek social support from chatbots, they disclose their situation gradually, yet most evaluations of supportive LLMs rely on single-turn, fully specified prompts. We introduce a multi-turn simulation framework that closes this gap. Support-seeking narratives from five Reddit communities are decomposed into ordered fragments and revealed turn by turn to a language model. Each response is coded with the Social Support Behavior Code (SSBC), an established multi-label taxonomy that captures the composition of support, rather than a single quality score. To ask whether support choices track the model's own construal of user distress, we use linear probes on hidden representations to estimate this internal signal without altering the generation context. Across two mid-scale models (Llama-3.1-8B, OLMo-3-7B) and more than 6,200 turns, support composition shifts systematically with estimated distress: teaching declines as estimated distress rises, a finding that replicates across architectures, while increases in affective and esteem-oriented strategies (such as validation) are suggestive but model-specific and rest on noisier annotations. Community context independently shapes behavior, tracking topic and discourse norms rather than demographic categories. These trajectory-level dynamics, invisible to single-turn evaluation, motivate multi-turn auditing frameworks for socially sensitive applications.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 47 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

12 of 47 references.

References · 12

  1. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  2. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  3. 2021Towards Emotional Support Dialog Systemscited by 294
  4. 2020A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Supportcited by 182
  5. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 144
  6. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  7. 2025SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants?cited by 45
  8. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 43
  9. 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
  10. 2026Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluationscited by 36
  11. 2025Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarioscited by 25
  12. 2024Towards a Client-Centered Assessment of LLM Therapists by Client Simulationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.