mlsanityData to 7 Oct 2026

Evaluating Language Model Agency through Negotiations

arXiv 2401.04536 ↗First posted 9 Jan 2024cs.CL cs.AI cs.LGcited by 43

Not on the map. Why?

Cited by 23 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce an approach to evaluate language model (LM) agency using negotiation games. This approach better reflects real-world use cases and addresses some of the shortcomings of alternative LM benchmarks. Negotiation games enable us to study multi-turn, and cross-model interactions, modulate complexity, and side-step accidental evaluation data leakage. We use our approach to test six widely used and publicly accessible LMs, evaluating performance and alignment in both self-play and cross-play settings. Noteworthy findings include: (i) only closed-source models tested here were able to complete these tasks; (ii) cooperative bargaining games proved to be most challenging to the models; and (iii) even the most powerful models sometimes "lose" to weaker opponents

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 44 of its 84 references that are in the map. First posted on arXiv 9 Jan 2024.

Builds on

44 of 84 references.

References · 44

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
32 more
  1. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  4. 2022Red Teaming Language Models with Language Modelscited by 983
  5. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  6. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  7. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  8. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  9. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  10. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  11. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  12. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  13. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  14. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  15. 2023How is ChatGPT's behavior changing over time?cited by 265
  16. 2023Reward Design with Language Modelscited by 257
  17. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  18. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  19. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 202
  20. 2020Open Problems in Cooperative AIcited by 167
  21. 2018Decoupling Strategy and Generation in Negotiation Dialoguescited by 156
  22. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  23. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  24. 2022Language Models as Agent Modelscited by 131
  25. 2023Mindstorms in Natural Language-Based Societies of Mindcited by 105
  26. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  27. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 89
  28. 2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 58
  29. 2020Human-Level Performance in No-Press Diplomacy via Equilibrium Searchcited by 41
  30. 2023Thought Cloning: Learning to Think while Acting by Imitating Human Thinkingcited by 30
  31. 2023Flows: Building Blocks of Reasoning and Collaborating AIcited by 25
  32. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 20; 2026: 14.

43 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 120
  3. 2024Emergent social conventions and collective bias in LLM populationscited by 115
  4. 2024How Well Can LLMs Negotiate? NegotiationArena Platform and Analysiscited by 93
  5. 2024Prediction-Powered Ranking of Large Language Modelscited by 29
  6. 2025TextArenacited by 24
  7. 2024STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Makingcited by 22
  8. 2024LLMs with Personalities in Multi-issue Negotiation Gamescited by 21
  9. 2024Self-Recognition in Language Modelscited by 17
  10. 2025Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontierscited by 14
  11. 2025The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperationcited by 11
  12. 2024PLAYER*: Enhancing LLM-based Multi-Agent Communication and Interaction in Murder Mystery Gamescited by 10
31 more
  1. 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applicationscited by 10
  2. 2024How Personality Traits Influence Negotiation Outcomes? A Simulation based on Large Language Modelscited by 9
  3. 2025The Agentic Economycited by 9
  4. 2025ARIA: Training Language Agents with Intention-Driven Reward Aggregationcited by 9
  5. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  6. 2025HARBOR: Exploring Persona Dynamics in Multi-Agent Competitioncited by 6
  7. 2025Agent-to-Agent Theory of Mind: Testing Interlocutor Awareness among Large Language Modelscited by 5
  8. 2025CORE: Measuring Multi-Agent LLM Interaction Quality under Game-Theoretic Pressurescited by 5
  9. 2025Evaluating Multi-Turn Bargain Skills in LLM-Based Seller Agentcited by 4
  10. 2026Reasoning-Driven Synthetic Data Generation and Evaluationcited by 4
  11. 2026Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewardscited by 4
  12. 2026Learning to Negotiate: Multi-Agent Deliberation for Collective Value Alignment in LLMscited by 3
  13. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  14. 2026Interactive Evaluation Requires a Design Sciencecited by 3
  15. 2025Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMscited by 2
  16. 2025GTAlign: Game-Theoretic Alignment of LLM Assistants for Social Welfarecited by 2
  17. 2025ProMediate: A Socio-cognitive framework for evaluating proactive agents in multi-party negotiationcited by 2
  18. 2025Learning Robust Social Strategies with Large Language Modelscited by 2
  19. 2025Simulating Persuasive Dialogues on Meat Reduction with Generative Agentscited by 1
  20. 2026PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarioscited by 1
  21. 2026Talk is Cheap, Communication is Hard: Dynamic Grounding Failures and Repair in Multi-Agent Negotiationcited by 1
  22. 2025Information Bargaining: Bilateral Commitment in Bayesian Persuasioncited by 0
  23. 2025LLMs for Resource Allocation: A Participatory Budgeting Approach to Inferring Preferencescited by 0
  24. 2025How Far Can LLMs Emulate Human Behavior?: A Strategic Analysis via the Buy-and-Sell Negotiation Gamecited by 0
  25. 2026[Re] Benchmarking LLM Capabilities in Negotiation through Scoreable Gamescited by 0
  26. 2026From Overload to Convergence: Supporting Multi-Issue Human-AI Negotiation with Bayesian Visualizationcited by 0
  27. 2026Counterparty Modeling is Not Strategy: The Limits of LLM Negotiatorscited by 0
  28. 2026Evaluating Rational Contracting in Natural Languagecited by 0
  29. 2026Do LLM Agents Negotiate Rationally? A Mechanism-Design Framework for Verifiable Multi-Agent Interaction over A2A/MCPcited by 0
  30. 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0
  31. 2026ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediationcited by 0