Cattle Trade: A Multi-Agent Benchmark for LLM Bluffing, Bidding, and Bargaining
On the map: LLM reasoning → Reasoning for agents, planning and code
also in 1 more
Abstract (arXiv)
We introduce textscCattle Trade, a multi-agent benchmark for evaluating large language models (LLMs) as agents in strategic reasoning under imperfect information, adversarial interaction, and resource constraints. The benchmark combines auctions, hidden-offer trade challenges (TCs), bargaining, bluffing, opponent modeling, and resource allocation within a single long-horizon game lasting 50--60 turns. Unlike prior agent benchmarks that test these abilities in isolation, textscCattle Trade evaluates whether agents integrate them across a competitive, multi-agent economic game with conflicting incentives. The benchmark logs every bid, TC offer, counteroffer, and card selection, enabling behavioural analysis beyond final scores or win rates. We evaluate seven cost-efficient language models and three deterministic code agents across 242 games. Strategic coherence, in particular spending efficiency, resource discipline, and phase-adaptive bidding, is associated with rank more strongly than spending volume or any single subskill. Two heuristic code agents outperform most tested LLMs, and behavioural traces surface recurring LLM failure modes including overbidding, self-bidding, bankrupt TC initiation, and weak opponent-state adaptation. Evaluating agentic competence requires benchmarks that test the joint deployment of multiple capabilities in multi-agent environments with conflicting incentives, uncertainty, and economic dynamics.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 16 of its 32 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
16 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2025LLM Strategic Reasoning: Agentic Study through Behavioral Game Theory27 Feb
- 2024GAMEBoT: Transparent Assessment of LLM Reasoning in Games18 Dec
- 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents7 Jun
- 2024GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations19 Feb
- 2023Strategic Reasoning with Language Models30 May
Other references · 11
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2023Exploring Large Language Models for Communication Games: An Empirical Study on Werewolfcited by 238
- 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
- 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 67
- 2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 54
- 2023AvalonBench: Evaluating LLMs Playing the Game of Avaloncited by 45
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026FM-Bench: A Benchmark for Long-Horizon Management with Competing Agentscited by 1
- 2026Do LLM Agents Negotiate Rationally? A Mechanism-Design Framework for Verifiable Multi-Agent Interaction over A2A/MCPcited by 0
- 2026FromPitch2Board: Benchmarking LLM Agents in Long-Horizon Football Managementcited by 0
- 2026FromPitch2Board: Benchmarking LLM Agents in Long-Horizon Football Management28 Sep 2026
- 2026FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents19 Aug 2026
- 2026Do LLM Agents Negotiate Rationally? A Mechanism-Design Framework for Verifiable Multi-Agent Interaction over A2A/MCP10 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026GENSTRAT: Toward a Science of Strategic Reasoning in Large Language Models
- 2025The Illusion of Rationality: Tacit Bias and Strategic Dominance in Frontier LLM Negotiation Games
- 2026TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate
- 2023Put Your Money Where Your Mouth Is: Evaluating Strategic Planning and Execution of LLM Agents in an Auction Arena
- 2025Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining
- 2026Training Language Models for Bilateral Trade with Private Information
- 2026SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game
- 2024STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making