# Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues

arXiv: https://arxiv.org/abs/2402.13550 · HTML: https://mlsanity.com/paper/2402.13550/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [Decoupling Strategy and Generation in Negotiation Dialogues](https://mlsanity.com/paper/1808.09637/) (2018)
- [Self-Discover: Large Language Models Self-Compose Reasoning Structures](https://mlsanity.com/paper/2402.03620/) (2024)
- [CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems](https://mlsanity.com/paper/2103.15721/) (2021)
- [A Survey of the Evolution of Language Model-Based Dialogue Systems: Data, Task and Models](https://mlsanity.com/paper/2311.16789/) (2023)
- [Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness](https://mlsanity.com/paper/2004.09731/) (2020)
- [Be Selfish, But Wisely: Investigating the Impact of Agent Personality in Mixed-Motive Human-Agent Interactions](https://mlsanity.com/paper/2310.14404/) (2023)
- [Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks](https://mlsanity.com/paper/2210.05664/) (2022)
- [Opponent Modeling in Negotiation Dialogues by Related Data Adaptation](https://mlsanity.com/paper/2205.00344/) (2022)
- [Towards Emotion-Aware Agents For Negotiation Dialogues](https://mlsanity.com/paper/2107.13165/) (2021)

## Built on by (most cited first, 29 of 29)

- [Strength Lies in Differences! Improving Strategy Planning for Non-collaborative Dialogues via Diversified User Simulation](https://mlsanity.com/paper/2403.06769/) (2024)
- [Beyond Nash Equilibrium: Bounded Rationality of LLMs and humans in Strategic Decision-making](https://mlsanity.com/paper/2506.09390/) (2025)
- [Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiences](https://mlsanity.com/paper/2510.06105/) (2025)
- [Digital Player: Evaluating Large Language Models based Human-like Agent in Games](https://mlsanity.com/paper/2502.20807/) (2025)
- [Advancing AI Negotiations: A Large-Scale Autonomous Negotiation Competition](https://mlsanity.com/paper/2503.06416/) (2025)
- [Real-Time Deadlines Reveal Fragile Temporal Adaptation in LLM Strategic Dialogues](https://mlsanity.com/paper/2601.13206/) (2026)
- [Fine-Grained and Thematic Evaluation of LLMs in Social Deduction Game](https://mlsanity.com/paper/2408.09946/) (2024)
- [Leveraging Large Language Models for Active Merchant Non-player Characters](https://mlsanity.com/paper/2412.11189/) (2024)
- [ASTRA: A Negotiation Agent with Adaptive and Strategic Reasoning via Tool-integrated Action for Dynamic Offer Optimization](https://mlsanity.com/paper/2503.07129/) (2025)
- [LLM Agents for Bargaining with Utility-based Feedback](https://mlsanity.com/paper/2505.22998/) (2025)
- [NomicLaw: Emergent Trust and Strategic Argumentation in LLMs During Collaborative Law-Making](https://mlsanity.com/paper/2508.05344/) (2025)
- [Personality Expression Across Contexts: Linguistic and Behavioral Variation in LLM Agents](https://mlsanity.com/paper/2602.01063/) (2026)
- [Estimating the Empowerment of Language Model Agents](https://mlsanity.com/paper/2509.22504/) (2025)
- [The Power of Stories: Narrative Priming Shapes How LLM Agents Collaborate and Compete](https://mlsanity.com/paper/2505.03961/) (2025)
- [Shall We Play a Game? Language Models for Open-ended Wargames](https://mlsanity.com/paper/2509.17192/) (2025)
- [Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy](https://mlsanity.com/paper/2512.18292/) (2025)
- [MERIT Feedback Elicits Better Bargaining in LLM Negotiators](https://mlsanity.com/paper/2602.10467/) (2026)
- [Emotionally-Aware Agents for Dispute Resolution](https://mlsanity.com/paper/2509.04465/) (2025)
- [PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios](https://mlsanity.com/paper/2602.05302/) (2026)
- [Diversity Without Fidelity: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation Simulation](https://mlsanity.com/paper/2604.11840/) (2026)
- [EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus](https://mlsanity.com/paper/2510.12899/) (2025)
- [The Language of Bargaining: Linguistic Effects in LLM Negotiations](https://mlsanity.com/paper/2601.04387/) (2026)
- [Communicate-Predict-Act: Evaluating Social Intelligence of Agents](https://mlsanity.com/paper/2604.08727/) (2026)
- [CoSToM:Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models](https://mlsanity.com/paper/2604.10031/) (2026)
- [Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling](https://mlsanity.com/paper/2605.12411/) (2026)
- [SoCRATES: Towards Reliable Automated Evaluation of Proactive LLM Mediation across Domains and Socio-cognitive Variations](https://mlsanity.com/paper/2606.05563/) (2026)
- [TradeVerse: A Longitudinal Benchmark of Political Negotiation in International Trade](https://mlsanity.com/paper/2608.06549/) (2026)
- [Deal Me Maybe: The Role of Emotions in Multi-Agent Negotiation](https://mlsanity.com/paper/2608.06922/) (2026)
- [ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation](https://mlsanity.com/paper/2609.11101/) (2026)

## Abstract (arXiv)

A successful negotiation requires a range of capabilities, including comprehension of the conversation context, Theory-of-Mind (ToM) skills to infer the partner's motives, strategic reasoning, and effective communication, making it challenging for automated systems. Despite the remarkable performance of LLMs in various NLP tasks, there is no systematic evaluation of their capabilities in negotiation. Such an evaluation is critical for advancing AI negotiation agents and negotiation research, ranging from designing dialogue systems to providing pedagogical feedback and scaling up data collection practices. This work aims to systematically analyze the multifaceted capabilities of LLMs across diverse dialogue scenarios throughout the stages of a typical negotiation interaction. Our analysis highlights GPT-4's superior performance in many tasks while identifying specific challenges, such as making subjective assessments and generating contextually appropriate, strategically advantageous responses.