# Determinants of LLM-assisted Decision-Making

arXiv: https://arxiv.org/abs/2402.17385 · HTML: https://mlsanity.com/paper/2402.17385/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 205 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 30 of 68)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023)

## Built on by (most cited first, 30 of 56)

- [AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways](https://mlsanity.com/paper/2406.02630/) (2024)
- [INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent](https://mlsanity.com/paper/2412.18174/) (2024)
- [VideoWorld: Exploring Knowledge Learning from Unlabeled Videos](https://mlsanity.com/paper/2501.09781/) (2025)
- [A Comprehensive Evaluation of Cognitive Biases in LLMs](https://mlsanity.com/paper/2410.15413/) (2024)
- [Large Language Models for UAVs: Current State and Pathways to the Future](https://mlsanity.com/paper/2405.01745/) (2024)
- [PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning](https://mlsanity.com/paper/2511.11562/) (2025)
- [Evaluating Human-AI Collaboration: A Review and Methodological Framework](https://mlsanity.com/paper/2407.19098/) (2024)
- [Towards Trustworthy AI: A Review of Ethical and Robust Large Language Models](https://mlsanity.com/paper/2407.13934/) (2024)
- [The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models](https://mlsanity.com/paper/2406.11096/) (2024)
- [Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets](https://mlsanity.com/paper/2510.25779/) (2025)
- [Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM Multi-Agent Systems](https://mlsanity.com/paper/2502.11098/) (2025)
- [ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes](https://mlsanity.com/paper/2403.06294/) (2024)
- [Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models](https://mlsanity.com/paper/2407.11282/) (2024)
- [A Survey on Human-Centric LLMs](https://mlsanity.com/paper/2411.14491/) (2024)
- [VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values](https://mlsanity.com/paper/2407.03000/) (2024)
- [AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support](https://mlsanity.com/paper/2504.06771/) (2025)
- [DecisionFlow: Advancing Large Language Model as Principled Decision Maker](https://mlsanity.com/paper/2505.21397/) (2025)
- [Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making](https://mlsanity.com/paper/2503.18419/) (2025)
- [CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcare](https://mlsanity.com/paper/2512.11437/) (2025)
- [LREF: A Novel LLM-based Relevance Framework for E-commerce](https://mlsanity.com/paper/2503.09223/) (2025)
- [A Survey on Enhancing Causal Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2503.09326/) (2025)
- [STRUX: An LLM for Decision-Making with Structured Explanations](https://mlsanity.com/paper/2410.12583/) (2024)
- [DVM: Towards Controllable LLM Agents in Social Deduction Games](https://mlsanity.com/paper/2501.06695/) (2025)
- [RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarms](https://mlsanity.com/paper/2507.01378/) (2025)
- [SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot](https://mlsanity.com/paper/2412.05187/) (2024)
- [HARBOR: Exploring Persona Dynamics in Multi-Agent Competition](https://mlsanity.com/paper/2502.12149/) (2025)
- [Beyond Correctness: Exposing LLM-generated Logical Flaws in Reasoning via Multi-step Automated Theorem Proving](https://mlsanity.com/paper/2512.23511/) (2025)
- [Enhancing Trust in Large Language Models via Uncertainty-Calibrated Fine-Tuning](https://mlsanity.com/paper/2412.02904/) (2024)
- [Systematic Bias in Large Language Models: Discrepant Response Patterns in Binary vs. Continuous Judgment Tasks](https://mlsanity.com/paper/2504.19445/) (2025)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)

## Abstract (arXiv)

Decision-making is a fundamental capability in everyday life. Large Language Models (LLMs) provide multifaceted support in enhancing human decision-making processes. However, understanding the influencing factors of LLM-assisted decision-making is crucial for enabling individuals to utilize LLM-provided advantages and minimize associated risks in order to make more informed and better decisions. This study presents the results of a comprehensive literature analysis, providing a structural overview and detailed analysis of determinants impacting decision-making with LLM support. In particular, we explore the effects of technological aspects of LLMs, including transparency and prompt engineering, psychological factors such as emotions and decision-making styles, as well as decision-specific determinants such as task difficulty and accountability. In addition, the impact of the determinants on the decision-making process is illustrated via multiple application scenarios. Drawing from our analysis, we develop a dependency framework that systematizes possible interactions in terms of reciprocal interdependencies between these determinants. Our research reveals that, due to the multifaceted interactions with various determinants, factors such as trust in or reliance on LLMs, the user's mental model, and the characteristics of information processing are identified as significant aspects influencing LLM-assisted decision-making processes. Our findings can be seen as crucial for improving decision quality in human-AI collaboration, empowering both users and organizations, and designing more effective LLM interfaces. Additionally, our work provides a foundation for future empirical investigations on the determinants of decision-making assisted by LLMs.