# Learning to Sell: Reinforcement Learning for Strategic Large Language Model Agents in Multi-Product Markets

arXiv: https://arxiv.org/abs/2609.33289 · HTML: https://mlsanity.com/paper/2609.33289/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 17 of 17)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [AgentTuning: Enabling Generalized Agent Abilities for LLMs](https://mlsanity.com/paper/2310.12823/) (2023)
- [Decoupling Strategy and Generation in Negotiation Dialogues](https://mlsanity.com/paper/1808.09637/) (2018)
- [Hierarchical Text Generation and Planning for Strategic Dialogue](https://mlsanity.com/paper/1712.05846/) (2017)
- [Be Selfish, But Wisely: Investigating the Impact of Agent Personality in Mixed-Motive Human-Agent Interactions](https://mlsanity.com/paper/2310.14404/) (2023)
- [Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design](https://mlsanity.com/paper/2301.13734/) (2023)
- [Efficient Multi-Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/2408.08706/) (2024)
- [Inventory Balancing with Online Learning](https://mlsanity.com/paper/1810.05640/) (2018)
- [Efficient Policy Evaluation with Safety Constraint for Reinforcement Learning](https://mlsanity.com/paper/2410.05655/) (2024)
- [Offline Two-Player Zero-Sum Markov Games with KL Regularization](https://mlsanity.com/paper/2605.13025/) (2026)
- [Training Language Models for Bilateral Trade with Private Information](https://mlsanity.com/paper/2604.16472/) (2026)
- [Algorithms for Online Matching, Assortment, and Pricing with Tight Weight-dependent Competitive Ratios](https://mlsanity.com/paper/1905.04770/) (2019)
- [Optimal Pricing of Information](https://mlsanity.com/paper/2102.13289/) (2021)

## Built on by (most cited first, 1 of 1)

- [Pessimistic Minimax Learning for Public-Private Information Games under Unilateral Coverage](https://mlsanity.com/paper/2610.04997/) (2026)

## Abstract (arXiv)

Autonomous large language model (LLM) agents operating in multi-product markets must make sequential decisions under information asymmetry and resource constraints. We develop a machine learning approach for training such agents to act effectively as sellers in a multi-item bargaining environment, where a seller concurrently negotiates a catalog of substitutable assets across a pool of independent buyers. Buyers hold private, heterogeneous valuations across products, and each can purchase at most one item. Facing limits on total communication turns, the seller must dynamically match buyers with the most profitable products considering their private valuations, while strategically allocating its limited interaction budget toward combinations of greater potential value. We formalize this problem as a Partially Observable Markov Decision Process using a structured, four-part message protocol that maps natural language into a parsable and regulated decision space. Using this formalization, we design a post-training method using Reinforcement Learning from Verifiable Rewards (RLVR). To evaluate this framework, we construct a multidimensional metric suite that quantifies constraint adherence, seller surplus extraction, and allocation quality. Our trained seller agent learns to match limited inventory to buyers more effectively, matching or outperforming trillion-parameter frontier models in both seller surplus extraction and buyer-product allocation quality. Finally, these learned strategies generalize robustly to unseen market structures, correlated valuation distributions, and price ranges not encountered during training.