mlsanityData to 7 Oct 2026

Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making

arXiv 2602.11924 ↗First posted 12 Feb 2026cs.HC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

LLMs are increasingly supporting decision-making across high-stakes domains, requiring critical reflection on the socio-technical factors that shape how humans and LLMs are assigned roles and interact during human-in-the-loop decision-making. This paper introduces the concept of human-LLM archetypes -- defined as re-curring socio-technical interaction patterns that structure the roles of humans and LLMs in collaborative decision-making. We describe 17 human-LLM archetypes derived from a scoping literature review and thematic analysis of 113 LLM-supported decision-making papers. Then, we evaluate these diverse archetypes across real-world clinical diagnostic cases to examine the potential effects of adopting distinct human-LLM archetypes on LLM outputs and decision outcomes. Finally, we present relevant tradeoffs and design choices across human-LLM archetypes, including decision control, social hierarchies, cognitive forcing strategies, and information requirements. Through our analysis, we show that selection of human-LLM interaction archetype can influence LLM outputs and decisions, bringing important risks and considerations for the designers of human-AI decision-making systems

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 130 of its 253 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

130 of 253 references.

References · 130

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2024A Survey on LLM-as-a-Judgecited by 1,384
  6. 2021Ethical and social risks of harm from Language Modelscited by 957
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  8. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  9. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
  10. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  11. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
  12. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
38 more
  1. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  2. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  3. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 216
  4. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  5. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  6. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  7. 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 169
  8. 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 153
  9. 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 137
  10. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  11. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  12. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  13. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  14. 2024What Large Language Models Know and What People Think They Knowcited by 109
  15. 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 97
  16. 2023Perspectives on Large Language Models for Relevance Judgmentcited by 84
  17. 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
  18. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  19. 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
  20. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  21. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  22. 2019Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?cited by 64
  23. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  24. 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
  25. 2024MedExQA: Medical Question Answering Benchmark with Multiple Explanationscited by 58
  26. 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
  27. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  28. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  29. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  30. 2024LLM Voting: Human Choices and AI Collective Decision Makingcited by 47
  31. 2020Automated Lay Language Summarization of Biomedical Scientific Reviewscited by 46
  32. 2024Personalized Large Language Modelscited by 44
  33. 2024CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendationcited by 44
  34. 2024Understanding User Experience in Large Language Model Interactionscited by 43
  35. 2024AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationcited by 43
  36. 2025LLMs for Explainable AI: A Comprehensive Surveycited by 42
  37. 2023Opportunities and Risks of LLMs for Scalable Deliberation with Poliscited by 41
  38. 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistenciescited by 39

50 most cited of 130 All 130 (JSON)

Built on by

Not cited yet.