# PrefBench: Evaluating Zero-Shot LLM Agents in Hidden-Preference Personalized Pricing Negotiations

arXiv: https://arxiv.org/abs/2605.22855 · HTML: https://mlsanity.com/paper/2605.22855/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 19 May 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 10 of 10)

- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding](https://mlsanity.com/paper/2404.13627/) (2024)
- [Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method](https://mlsanity.com/paper/2402.15813/) (2024)
- [Model Distillation for Revenue Optimization: Interpretable Personalized Pricing](https://mlsanity.com/paper/2007.01903/) (2020)
- [Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locations](https://mlsanity.com/paper/1909.04760/) (2019)
- [Dynamic Pricing on E-commerce Platform with Deep Reinforcement Learning: A Field Experiment](https://mlsanity.com/paper/1912.02572/) (2019)
- [RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation](https://mlsanity.com/paper/2312.14095/) (2023)
- [Dynamic Pricing in High-Speed Railways Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.08234/) (2025)

## Abstract (arXiv)

Personalized pricing negotiations are a challenging testbed for LLM agents because successful interaction does not guarantee profitable decision making. A seller may produce valid actions and close many deals while still pricing poorly when buyer willingness to pay and bargaining traits remain hidden. This paper presents PrefBench, a simulator-based benchmark for hidden-preference personalized pricing negotiations. Each episode pairs a simulated buyer with a fixed vehicle-customization bundle; the seller observes public persona descriptors, bundle information, and negotiation history, while latent buyer variables govern valuation, patience, counter-offer behavior, and walkaway decisions. PrefBench evaluates this setting through an LLM-facing state-summary protocol that constrains agents to return strict JSON actions under a fixed hidden-information boundary. We evaluate zero-shot LLM sellers against heuristic references over 7,500 episodes. The tested LLMs follow the protocol reliably and achieve deal rates above 0.99, but their seller-profit outcomes remain weak: the best LLM average profit is only slightly above the random baseline and far below a simple concession heuristic under the same episode stream. These results show that structured action compliance and agreement-seeking behavior can coexist with weak profit-sensitive bargaining. PrefBench provides a controlled benchmark for evaluating pricing-agent behavior under hidden buyer preferences.