PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
We present PricingLogic, the first benchmark that probes whether Large Language Models(LLMs) can reliably automate tourism-related prices when multiple, overlapping fare rules apply. Travel agencies are eager to offload this error-prone task onto AI systems; however, deploying LLMs without verified reliability could result in significant financial losses and erode customer trust. PricingLogic comprises 300 natural-language questions based on booking requests derived from 42 real-world pricing policies, spanning two levels of difficulty: (i) basic customer-type pricing and (ii)bundled-tour calculations involving interacting discounts. Evaluations of a line of LLMs reveal a steep performance drop on the harder tier,exposing systematic failures in rule interpretation and arithmetic reasoning.These results highlight that, despite their general capabilities, today's LLMs remain unreliable in revenue-critical applications without further safeguards or domain adaptation. Our code and dataset are available at https://github.com/EIT-NLP/PricingLogic.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 19 of its 21 references that are in the map. First posted on arXiv 14 Oct 2025.
Builds on
19 of 21 references.
Closest ancestors · 1
Same direction, newest first
- 2024Qwen2.5-Coder Technical Report18 Sep
Other references · 18
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024GPT-4o System Cardcited by 4,467
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
- 2024DeepSeek-V3 Technical Reportcited by 724
- 2022PAL: Program-aided Language Modelscited by 635
- 2023Faithful Chain-of-Thought Reasoningcited by 285
- 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
6 more
- 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
- 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 112
- 2025SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?cited by 99
- 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 85
- 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoningcited by 34
- 2024RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarioscited by 26
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025LETToT: Label-Free Evaluation of Large Language Models On Tourism Using Expert Tree-of-Thought
- 2026PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
- 2026TravelEval: A Comprehensive Benchmarking Framework for Evaluating LLM-Powered Travel Planning Agents
- 2025LLP: LLM-Based Product Pricing in E-commerce
- 2026AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing
- 2026TourMart: A Parametric Audit Instrument for Commission Steering in LLM Travel Agents
- 2026EComStage: Stage-wise and Orientation-specific Benchmarking for Large Language Models in E-commerce
- 2025Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards