mlsanityData to 7 Oct 2026

PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks

arXiv 2510.12409 ↗First posted 14 Oct 2025cs.AIcited by 2

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

We present PricingLogic, the first benchmark that probes whether Large Language Models(LLMs) can reliably automate tourism-related prices when multiple, overlapping fare rules apply. Travel agencies are eager to offload this error-prone task onto AI systems; however, deploying LLMs without verified reliability could result in significant financial losses and erode customer trust. PricingLogic comprises 300 natural-language questions based on booking requests derived from 42 real-world pricing policies, spanning two levels of difficulty: (i) basic customer-type pricing and (ii)bundled-tour calculations involving interacting discounts. Evaluations of a line of LLMs reveal a steep performance drop on the harder tier,exposing systematic failures in rule interpretation and arithmetic reasoning.These results highlight that, despite their general capabilities, today's LLMs remain unreliable in revenue-critical applications without further safeguards or domain adaptation. Our code and dataset are available at https://github.com/EIT-NLP/PricingLogic.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 19 of its 21 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

19 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Qwen2.5-Coder Technical Report18 Sep

Other references · 18

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  7. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  8. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  9. 2024DeepSeek-V3 Technical Reportcited by 724
  10. 2022PAL: Program-aided Language Modelscited by 635
  11. 2023Faithful Chain-of-Thought Reasoningcited by 285
  12. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
6 more
  1. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  2. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 112
  3. 2025SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?cited by 99
  4. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 85
  5. 2024MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoningcited by 34
  6. 2024RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarioscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025LETToT: Label-Free Evaluation of Large Language Models On Tourism Using Expert Tree-of-Thought
  2. 2026PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
  3. 2026TravelEval: A Comprehensive Benchmarking Framework for Evaluating LLM-Powered Travel Planning Agents
  4. 2025LLP: LLM-Based Product Pricing in E-commerce
  5. 2026AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing
  6. 2026TourMart: A Parametric Audit Instrument for Commission Steering in LLM Travel Agents
  7. 2026EComStage: Stage-wise and Orientation-specific Benchmarking for Large Language Models in E-commerce
  8. 2025Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards