# ChinaTravel: An Open-Ended Travel Planning Benchmark with Compositional Constraint Validation for Language Agents

arXiv: https://arxiv.org/abs/2412.13682 · HTML: https://mlsanity.com/paper/2412.13682/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2024.

## Built on by (most cited first, 16 of 16)

- [Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2508.13678/) (2025)
- [Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe](https://mlsanity.com/paper/2603.21972/) (2026)
- [Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling](https://mlsanity.com/paper/2604.08178/) (2026)
- [GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planning](https://mlsanity.com/paper/2605.25200/) (2026)
- [Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use](https://mlsanity.com/paper/2607.01084/) (2026)
- [Thinking with Tables: Enhancing Multi-Modal Tabular Understanding via Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2603.24004/) (2026)
- [Trip+: Benchmarking Agents in Personalized Interactive Travel Planning](https://mlsanity.com/paper/2606.21169/) (2026)
- [TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning](https://mlsanity.com/paper/2607.26977/) (2026)
- [Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models](https://mlsanity.com/paper/2608.20237/) (2026)
- [Bi-CoG: Bi-Consistency-Guided Self-Training for Vision-Language Models](https://mlsanity.com/paper/2510.20477/) (2025)
- [A Progressive Visual-Logic-Aligned Framework for Ride-Hailing Adjudication](https://mlsanity.com/paper/2603.17328/) (2026)
- [Revisiting the Travel Planning Capabilities of Large Language Models](https://mlsanity.com/paper/2605.03308/) (2026)
- [LifePlanner: Evaluating LLM Agents for Geo-spatial Planning with Social Media Data](https://mlsanity.com/paper/2608.25039/) (2026)
- [SymboLLM-FE: LLM-Accelerated Symbolic Regression for Automated Feature Engineering on Tabular Data](https://mlsanity.com/paper/2608.28408/) (2026)
- [UTP-Bench: Uncertainty-aware Travel Planning Benchmark](https://mlsanity.com/paper/2609.02421/) (2026)
- [BANGLAPLAN: A Benchmark for Constraint-Based Planning and Decision-Making in Bangla](https://mlsanity.com/paper/2610.19704/) (2026)

## Abstract (arXiv)

Travel planning stands out among real-world applications of Language Agents because it couples significant practical demand with a rigorous constraint-satisfaction challenge. However, existing benchmarks primarily operate on a slot-filling paradigm, restricting agents to synthetic queries with pre-defined constraint menus, which fails to capture the open-ended nature of natural language interaction, where user requirements are compositional, diverse, and often implicitly expressed. To address this gap, we introduce ChinaTravel, with four key contributions: 1) a practical sandbox aligned with the multi-day, multi-POI travel planning, 2) a compositionally generalizable domain-specific language (DSL) for scalable evaluation, covering feasibility, constraint satisfaction, and preference comparison 3) an open-ended dataset that integrates diverse travel requirements and implicit intent from 1154 human participants, and 4) fine-grained analysis reveal the potential of neuro-symbolic agents in travel planning, achieving a 37.0% constraint satisfaction rate on human queries, a 10 × improvement over purely neural models, yet highlighting significant challenges in compositional generalization. Overall, ChinaTravel provides a foundation for advancing language agents through compositional constraint validation in complex, real-world planning scenarios. Project Page: https://www.lamda.nju.edu.cn/shaojj/ChinaTravel/index.html