# Training Language Models for Bilateral Trade with Private Information

arXiv: https://arxiv.org/abs/2604.16472 · HTML: https://mlsanity.com/paper/2604.16472/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [ReFT: Reasoning with Reinforced Fine-Tuning](https://mlsanity.com/paper/2401.08967/) (2024)
- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023)
- [Decoupling Strategy and Generation in Negotiation Dialogues](https://mlsanity.com/paper/1808.09637/) (2018)
- [How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis](https://mlsanity.com/paper/2402.05863/) (2024)
- [Evaluating Language Model Agency through Negotiations](https://mlsanity.com/paper/2401.04536/) (2024)
- [Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method](https://mlsanity.com/paper/2402.15813/) (2024)
- [Improving Dialog Systems for Negotiation with Personality Modeling](https://mlsanity.com/paper/2010.09954/) (2020)
- [CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning](https://mlsanity.com/paper/2204.08426/) (2022)
- [Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner](https://mlsanity.com/paper/2406.11978/) (2024)
- [Multi-Issue Bargaining With Deep Reinforcement Learning](https://mlsanity.com/paper/2002.07788/) (2020)
- [AgreeMate: Teaching LLMs to Haggle](https://mlsanity.com/paper/2412.18690/) (2024)
- [FishBargain: An LLM-Empowered Bargaining Agent for Online Fleamarket Platform Sellers](https://mlsanity.com/paper/2502.10406/) (2025)
- [LLM Agents for Bargaining with Utility-based Feedback](https://mlsanity.com/paper/2505.22998/) (2025)

## Built on by (most cited first, 5 of 5)

- [Strategic Bargaining in Multi-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiations](https://mlsanity.com/paper/2607.05863/) (2026)
- [Learning to Sell: Reinforcement Learning for Strategic Large Language Model Agents in Multi-Product Markets](https://mlsanity.com/paper/2609.33289/) (2026)
- [From Passive Delegates to Strategic Negotiators: Reinforcing Social Reasoning in Small Language Models with SocialRL](https://mlsanity.com/paper/2608.13787/) (2026)
- [Do LLM Agents Negotiate Rationally? A Mechanism-Design Framework for Verifiable Multi-Agent Interaction over A2A/MCP](https://mlsanity.com/paper/2608.14613/) (2026)
- [Do Small Language Models Learn to Negotiate? A Controlled Scaling Study of RL-Trained Sellers](https://mlsanity.com/paper/2610.06204/) (2026)

## Abstract (arXiv)

Bilateral bargaining under incomplete information provides a controlled testbed for evaluating large language model (LLM) agent capabilities. Bilateral trade demands individual rationality, strategic surplus maximization, and cooperation to realize gains from trade. We develop a structured bargaining environment where LLMs negotiate via tool calls within an event-driven simulator, separating binding offers from natural-language messages to enable automated evaluation. The environment serves two purposes: as a benchmark for frontier models and as a training environment for open-weight models via reinforcement learning. In benchmark experiments, a round-robin tournament among five frontier models (15,000 negotiations) reveals that effective strategies implement price discrimination through sequential offers. Aggressive anchoring, calibrated concession, and temporal patience correlate with the highest surplus share and deal rate. Accommodating strategies that concede quickly disable price discrimination in the buyer role, yielding the lowest surplus capture and deal completion. Stronger models scale their behavior proportionally to item value, maintaining performance across price tiers; weaker models perform well only when wide zones of possible agreement offset suboptimal strategies. In training experiments, we fine-tune Qwen3 (8B, 14B) via supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO) against a fixed frontier opponent. These stages optimize competing objectives: SFT approximately doubles surplus share but reduces deal rates, while RL recovers deal rates but erodes surplus gains, reflecting the reward structure. SFT also compresses surplus variation across price tiers, which generalizes to unseen opponents, suggesting that behavioral cloning instills proportional strategies rather than memorized price points.