mlsanityData to 7 Oct 2026

LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users

arXiv 2406.17737 ↗First posted 25 Jun 2024cs.CL cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

While state-of-the-art large language models (LLMs) have shown impressive performance on many tasks, there has been extensive research on undesirable model behavior such as hallucinations and bias. In this work, we investigate how the quality of LLM responses changes in terms of information accuracy, truthfulness, and refusals depending on three user traits: English proficiency, education level, and country of origin. We present extensive experimentation on three state-of-the-art LLMs and two different datasets targeting truthfulness and factuality. Our findings suggest that undesirable behaviors in state-of-the-art LLMs occur disproportionately more for users with lower English proficiency, of lower education status, and originating from outside the US, rendering these models unreliable sources of information towards their most vulnerable users.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 11 of its 24 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

11 of 24 references.

References · 11

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  4. 2022Red Teaming Language Models with Language Modelscited by 983
  5. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  6. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  7. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  8. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  9. 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 71
  10. 2023When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviourcited by 61
  11. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

By area:

  1. 2025Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Stylescited by 15
  2. 2025Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generationcited by 10
  3. 2026One Persona, Many Cues, Different Results: How Sociodemographic Cues Impact LLM Personalizationcited by 9
  4. 2025Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Modelscited by 7
  5. 2025Characterizing Selective Refusal Bias in Large Language Modelscited by 4
  6. 2026From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequencescited by 3
  7. 2026Dialect vs Demographics: Quantifying LLM Bias from Implicit Linguistic Signals vs. Explicit User Profilescited by 3
  8. 2026Differential Harm Propensity in Personalized LLM Agents: The Curious Case of Mental Health Disclosurecited by 2
  9. 2025Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biasescited by 1
  10. 2026MIRA: A Bilingual Benchmark for Medical Information Response Auditcited by 1
  11. 2026Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answerscited by 1
  12. 2026Affective Context Amplifies Sycophancy in LLM Responsescited by 1
3 more
  1. 2025Cognitive-Level Adaptive Generation via Capability-Aware Retrieval and Style Adaptationcited by 0
  2. 2026Defeat Devices in AI Systemscited by 0
  3. 2026The complexities of patient-centred conversational artificial intelligencecited by 0