Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Not on the map. Why?
Abstract (arXiv)
LLMs are increasingly supporting decision-making across high-stakes domains, requiring critical reflection on the socio-technical factors that shape how humans and LLMs are assigned roles and interact during human-in-the-loop decision-making. This paper introduces the concept of human-LLM archetypes -- defined as re-curring socio-technical interaction patterns that structure the roles of humans and LLMs in collaborative decision-making. We describe 17 human-LLM archetypes derived from a scoping literature review and thematic analysis of 113 LLM-supported decision-making papers. Then, we evaluate these diverse archetypes across real-world clinical diagnostic cases to examine the potential effects of adopting distinct human-LLM archetypes on LLM outputs and decision outcomes. Finally, we present relevant tradeoffs and design choices across human-LLM archetypes, including decision control, social hierarchies, cognitive forcing strategies, and information requirements. Through our analysis, we show that selection of human-LLM interaction archetype can influence LLM outputs and decisions, bringing important risks and considerations for the designers of human-AI decision-making systems
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 130 of its 253 references that are in the map. First posted on arXiv 12 Feb 2026.
Builds on
130 of 253 references.
References · 130
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
- 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
- 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
- 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
- 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
38 more
- 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
- 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
- 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 216
- 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
- 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
- 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
- 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 169
- 2023Simulating Opinion Dynamics with Networks of LLM-based Agentscited by 153
- 2024Large language models that replace human participants can harmfully misportray and flatten identity groupscited by 137
- 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
- 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
- 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
- 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
- 2024What Large Language Models Know and What People Think They Knowcited by 109
- 2024Sycophancy in Large Language Models: Causes and Mitigationscited by 97
- 2023Perspectives on Large Language Models for Relevance Judgmentcited by 84
- 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
- 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
- 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
- 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
- 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
- 2019Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?cited by 64
- 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
- 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
- 2024MedExQA: Medical Question Answering Benchmark with Multiple Explanationscited by 58
- 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
- 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
- 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
- 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
- 2024LLM Voting: Human Choices and AI Collective Decision Makingcited by 47
- 2020Automated Lay Language Summarization of Biomedical Scientific Reviewscited by 46
- 2024Personalized Large Language Modelscited by 44
- 2024CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendationcited by 44
- 2024Understanding User Experience in Large Language Model Interactionscited by 43
- 2024AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationcited by 43
- 2025LLMs for Explainable AI: A Comprehensive Surveycited by 42
- 2023Opportunities and Risks of LLMs for Scalable Deliberation with Poliscited by 41
- 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistenciescited by 39
50 most cited of 130 All 130 (JSON)
Built on by
Not cited yet.