# GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planning

arXiv: https://arxiv.org/abs/2605.25200 · HTML: https://mlsanity.com/paper/2605.25200/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 102 references that are in the map.
- First posted on arXiv 24 May 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [Ask-before-Plan: Proactive Language Agents for Real-World Planning](https://mlsanity.com/paper/2406.12639/) (2024)
- [Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Tools](https://mlsanity.com/paper/2404.11891/) (2024)
- [DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraints](https://mlsanity.com/paper/2601.18137/) (2026)
- [TripTailor: A Real-World Benchmark for Personalized Travel Planning](https://mlsanity.com/paper/2508.01432/) (2025)
- [DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for Autonomous Travel Planning Agents](https://mlsanity.com/paper/2509.21842/) (2025)
- [ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking](https://mlsanity.com/paper/2601.06487/) (2026)
- [ChinaTravel: An Open-Ended Travel Planning Benchmark with Compositional Constraint Validation for Language Agents](https://mlsanity.com/paper/2412.13682/) (2024)
- [TP-RAG: Benchmarking Retrieval-Augmented Large Language Model Agents for Spatiotemporal-Aware Travel Planning](https://mlsanity.com/paper/2504.08694/) (2025)
- [RETAIL: Towards Real-world Travel Planning for Large Language Models](https://mlsanity.com/paper/2508.15335/) (2025)
- [TripTide: A Benchmark for Adaptive Travel Planning under Disruptions](https://mlsanity.com/paper/2510.21329/) (2025)
- [Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe](https://mlsanity.com/paper/2603.21972/) (2026)
- [Wide-Horizon Thinking and Simulation-Based Evaluation for Real-World LLM Planning with Multifaceted Constraints](https://mlsanity.com/paper/2506.12421/) (2025)
- [TourPlanner: A Competitive Consensus Framework with Constraint-Gated Reinforcement Learning for Travel Planning](https://mlsanity.com/paper/2601.04698/) (2026)
- [WorldTravel: A Realistic Multimodal Travel-Planning Benchmark with Tightly Coupled Constraints](https://mlsanity.com/paper/2602.08367/) (2026)
- [Optimizing Group Utility in Itinerary Planning: A Strategic and Crowd-Aware Approach](https://mlsanity.com/paper/2304.08495/) (2023)
- [ODUTQA-MDC: A Task for Open-Domain Underspecified Tabular QA with Multi-turn Dialogue-based Clarification](https://mlsanity.com/paper/2604.10159/) (2026)
- [Decoupled Travel Planning with Behavior Forest](https://mlsanity.com/paper/2604.21354/) (2026)

## Built on by (most cited first, 5 of 5)

- [TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning](https://mlsanity.com/paper/2607.26977/) (2026)
- [Multi-Stakeholder LLM Alignment: Decomposing Estimation from Aggregation](https://mlsanity.com/paper/2605.26878/) (2026)
- [AI Tour Meeting: Group Travel Planning by LLM Agents](https://mlsanity.com/paper/2607.18806/) (2026)
- [Consilience: Conformally Calibrated Communication Control for Hidden-Profile Multi-Agent Reasoning](https://mlsanity.com/paper/2608.20564/) (2026)
- [PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents](https://mlsanity.com/paper/2609.34930/) (2026)

## Abstract (arXiv)

Travel planning in the real world is overwhelmingly a textitgroup activity, yet existing LLM travel-planning benchmarks reduce it to a single user, where the field is approaching saturation. This single-user assumption sidesteps what makes group planning hard for an agent: discovering private preferences across multiple users, surfacing conflicts, and balancing utility against fairness. To bring the task back to its multi-user reality, we introduce textbftextitGroupTravelBench, the first benchmark for multi-user, multi-turn travel planning. Built from real user profiles, POI data, and ticket prices, it comprises 650 tasks across three difficulty levels, each running in a synchronous group-chat sandbox with cached tool data for reproducible offline evaluation. Beyond the multi-step reasoning and tool use that single-user benchmarks already test, GroupTravelBench probes three group-specific capabilities: textit(i) elicitation of private preferences through multi-turn dialogue; textit(ii) coordination of inter-user conflicts via compromise or subgrouping; and textit(iii) planning that balances group utility against fairness. We pair this with a complementary evaluation framework combining rule-based outcome metrics and LLM-judge process metrics. Across a wide range of frontier models, even the strongest agents fall short on all four rule-based outcome metrics, with plan validity below 12%, suggesting that group-level outcome quality is a key open challenge for LLM travel-planning agents.