# ChoiceMates: Supporting Unfamiliar Online Decision-Making with Multi-Agent Conversational Interactions

arXiv: https://arxiv.org/abs/2310.01331 · HTML: https://mlsanity.com/paper/2310.01331/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 26 of its 117 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 26 of 26)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents](https://mlsanity.com/paper/2306.03314/) (2023)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [A Survey on Conversational Recommender Systems](https://mlsanity.com/paper/2004.00646/) (2020)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support](https://mlsanity.com/paper/2307.15810/) (2023)
- [CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes](https://mlsanity.com/paper/2308.06921/) (2023)
- [Enabling Conversational Interaction with Mobile UI using Large Language Models](https://mlsanity.com/paper/2209.08655/) (2022)
- [Graphologue: Exploring Large Language Model Responses with Interactive Diagrams](https://mlsanity.com/paper/2305.11473/) (2023)
- [Leveraging Large Language Models in Conversational Recommender Systems](https://mlsanity.com/paper/2305.07961/) (2023)
- [ChatGPT: A Meta-Analysis after 2.5 Months](https://mlsanity.com/paper/2302.13795/) (2023)
- [Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models](https://mlsanity.com/paper/2305.11483/) (2023)
- [Writer-Defined AI Personas for On-Demand Feedback Generation](https://mlsanity.com/paper/2309.10433/) (2023)
- [The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems](https://mlsanity.com/paper/2008.09100/) (2020)
- [Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions](https://mlsanity.com/paper/1905.10700/) (2019)
- [Prompting a Large Language Model to Generate Diverse Motivational Messages: A Comparison with Human-Written Messages](https://mlsanity.com/paper/2308.13479/) (2023)
- [Conversational Recommendation: A Grand AI Challenge](https://mlsanity.com/paper/2203.09126/) (2022)
- [Investigating Conversational Search Behavior For Domain Exploration](https://mlsanity.com/paper/2301.04098/) (2023)
- [Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making](https://mlsanity.com/paper/2202.02175/) (2022)
- [Inform the uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot](https://mlsanity.com/paper/2302.00832/) (2023)
- [What-if Analysis for Business Professionals: Current Practices and Future Opportunities](https://mlsanity.com/paper/2212.13643/) (2022)

## Built on by (most cited first, 22 of 22)

- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents](https://mlsanity.com/paper/2412.03563/) (2024)
- [AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation](https://mlsanity.com/paper/2402.14978/) (2024)
- [REAPER: Reasoning based Retrieval Planning for Complex RAG Systems](https://mlsanity.com/paper/2407.18553/) (2024)
- [Multi-Agents are Social Groups: Investigating Social Influence of Multiple Agents in Human-Agent Interactions](https://mlsanity.com/paper/2411.04578/) (2024)
- [Optimal-Agent-Selection: State-Aware Routing Framework for Efficient Multi-Agent Collaboration](https://mlsanity.com/paper/2511.02200/) (2025)
- [GestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents](https://mlsanity.com/paper/2310.12821/) (2023)
- [Conversational Agents as Catalysts for Critical Thinking: Challenging Design Fixation in Group Design](https://mlsanity.com/paper/2406.11125/) (2024)
- [Conversational Agents as Catalysts for Critical Thinking: Challenging Social Influence in Group Decision-making](https://mlsanity.com/paper/2503.14263/) (2025)
- [Constella: Supporting Storywriters' Interconnected Character Creation through LLM-based Multi-Agents](https://mlsanity.com/paper/2507.05820/) (2025)
- [AI-Enhanced Sensemaking: Exploring the Design of a Generative AI-Based Assistant to Support Genetic Professionals](https://mlsanity.com/paper/2412.15444/) (2024)
- [Carbon and Silicon, Coexist or Compete? A Survey on Human-AI Interactions in Agent-based Modeling and Simulation](https://mlsanity.com/paper/2502.18145/) (2025)
- [Multi-Agent Debate Strategies: Survey, Taxonomy, and Challenges](https://mlsanity.com/paper/2607.26212/) (2026)
- [The Use of Multiple Conversational Agent Interlocutors in Learning](https://mlsanity.com/paper/2312.16534/) (2023)
- [Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectives](https://mlsanity.com/paper/2604.06091/) (2026)
- [Multi-Agent Systems Shape Social Norms for Prosocial Behavior Change](https://mlsanity.com/paper/2602.07433/) (2026)
- [More Isn't Always Better: Balancing Decision Accuracy and Conformity Pressures in Multi-AI Advice](https://mlsanity.com/paper/2603.22152/) (2026)
- [SimViews: An Interactive Multi-Agent System Simulating Visitor-to-Visitor Conversational Patterns to Present Diverse Perspectives of Artifacts in Virtual Museums](https://mlsanity.com/paper/2508.07730/) (2025)
- [Exploring Learners' Expectations and Engagement When Collaborating with Constructively Controversial Peer Agents](https://mlsanity.com/paper/2601.13479/) (2026)
- [When Nobody Around Is Real: Exploring Public Opinions and User Experiences On the Multi-Agent AI Social Platform](https://mlsanity.com/paper/2601.18275/) (2026)
- [Decisive: Guiding User Decisions with Optimal Preference Elicitation from Unstructured Documents](https://mlsanity.com/paper/2604.18122/) (2026)
- [DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents](https://mlsanity.com/paper/2609.29309/) (2026)

## Abstract (arXiv)

From deciding on a PhD program to buying a new camera, unfamiliar decisions--decisions without domain knowledge--are frequent and significant. The complexity and uncertainty of such decisions demand unique approaches to information seeking, understanding, and decision-making. Our formative study highlights that users want to start by discovering broad and relevant domain information evenly and simultaneously, quickly address emerging inquiries, and gain personalized standards to assess information found. We present ChoiceMates, an interactive multi-agent system designed to address these needs by enabling users to engage with a dynamic set of LLM agents each presenting a unique experience in the domain. Unlike existing multi-agent systems that automate tasks with agents, the user orchestrates agents to assist their decision-making process. Our user evaluation (n=12) shows that ChoiceMates enables a more confident, satisfactory decision-making with better situation understanding than web search, and higher decision quality and confidence than a commercial multi-agent framework. This work provides insights into designing a more controllable and collaborative multi-agent system.