# Beyond Itinerary Planning-A Real-World Benchmark for Multi-Turn and Tool-Using Travel Tasks

arXiv: https://arxiv.org/abs/2512.22673 · HTML: https://mlsanity.com/paper/2512.22673/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2025.

## Built on by (most cited first, 10 of 10)

- [Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe](https://mlsanity.com/paper/2603.21972/) (2026)
- [Trip+: Benchmarking Agents in Personalized Interactive Travel Planning](https://mlsanity.com/paper/2606.21169/) (2026)
- [WRIT: Write-Read Intensive Trajectory Synthesis for Multi-Turn User-Facing Agents](https://mlsanity.com/paper/2606.02908/) (2026)
- [TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning](https://mlsanity.com/paper/2607.26977/) (2026)
- [Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis](https://mlsanity.com/paper/2607.23524/) (2026)
- [Replan, Repair, or Edit? A Unified Empirical Evaluation of Travel Agents for Itinerary Revision under Resource Disruptions](https://mlsanity.com/paper/2609.19654/) (2026)
- [TourMart: A Parametric Audit Instrument for Commission Steering in LLM Travel Agents](https://mlsanity.com/paper/2605.10440/) (2026)
- [MapSatisfyBench: Benchmarking Satisfaction-Aware Map Agents through Behavior-Grounded Implicit Decision Factors](https://mlsanity.com/paper/2606.17453/) (2026)
- [LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Data](https://mlsanity.com/paper/2608.25039/) (2026)
- [Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms](https://mlsanity.com/paper/2609.27321/) (2026)

## Abstract (arXiv)

Travel planning is a natural real-world task to test large language models' (LLMs) planning and tool-use abilities. Although prior work has studied LLM performance on travel planning, existing settings still differ from real-world needs, mainly due to limited domain coverage, insufficient modeling of users' implicit preferences in multi-turn conversations, and a lack of evaluation of agents' capability boundaries. To mitigate these gaps, we propose TravelBench, a benchmark for textittruly real-world travel planning. We collect user queries, user preferences, and tools from real scenarios, and construct three subtasks -- textitSingle-Turn, textitMulti-Turn, and textitUnsolvable -- to evaluate agents' three core capabilities in real settings: (1) solving problems independently, (2) interacting with users to elicit implicit preferences, and (3) recognizing the capability boundaries. To enable stable tool invocation and reproducible evaluation, we cache real tool-call results and build a sandbox environment which integrates ten travel-related tools, enabling agents to combine these tools to solve most practical travel planning problems. We evaluate multiple LLMs on TravelBench and find that even advanced models exhibit imbalanced performance across different capabilities. Our further systematic verification demonstrates the stability of the proposed benchmark. TravelBench provides a practical and reproducible benchmark to advance research on LLM agents for real-world travel planning.