PrefBench: Evaluating Zero-Shot LLM Agents in Hidden-Preference Personalized Pricing Negotiations
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Personalized pricing negotiations are a challenging testbed for LLM agents because successful interaction does not guarantee profitable decision making. A seller may produce valid actions and close many deals while still pricing poorly when buyer willingness to pay and bargaining traits remain hidden. This paper presents PrefBench, a simulator-based benchmark for hidden-preference personalized pricing negotiations. Each episode pairs a simulated buyer with a fixed vehicle-customization bundle; the seller observes public persona descriptors, bundle information, and negotiation history, while latent buyer variables govern valuation, patience, counter-offer behavior, and walkaway decisions. PrefBench evaluates this setting through an LLM-facing state-summary protocol that constrains agents to return strict JSON actions under a fixed hidden-information boundary. We evaluate zero-shot LLM sellers against heuristic references over 7,500 episodes. The tested LLMs follow the protocol reliably and achieve deal rates above 0.99, but their seller-profit outcomes remain weak: the best LLM average profit is only slightly above the random baseline and far below a simple concession heuristic under the same episode stream. These results show that structured action compliance and agreement-seeking behavior can coexist with weak profit-sensitive bargaining. PrefBench provides a controlled benchmark for evaluating pricing-agent behavior under hidden buyer preferences.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 10 of its 25 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
10 of 25 references.
References · 10
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2024NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surroundingcited by 49
- 2024Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Methodcited by 44
- 2020Model Distillation for Revenue Optimization: Interpretable Personalized Pricingcited by 24
- 2019Deep Reinforcement Learning Algorithm for Dynamic Pricing of Express Lanes with Multiple Access Locationscited by 8
- 2019Dynamic Pricing on E-commerce Platform with Deep Reinforcement Learning: A Field Experimentcited by 6
- 2023RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluationcited by 5
- 2025Dynamic Pricing in High-Speed Railways Using Multi-Agent Reinforcement Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactions
- 2026PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
- 2026Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce
- 2026TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate
- 2026When LLM Agents Negotiate: Private Information and Dynamic Bargaining in Supply Chains
- 2024AgreeMate: Teaching LLMs to Haggle
- 2025The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets
- 2026Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information