# Is Your LLM-Based Multi-Agent a Reliable Real-World Planner? Exploring Fraud Detection in Travel Planning

arXiv: https://arxiv.org/abs/2505.16557 · HTML: https://mlsanity.com/paper/2505.16557/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 21 of 21)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [ADaPT: As-Needed Decomposition and Planning with Language Models](https://mlsanity.com/paper/2311.05772/) (2023)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents](https://mlsanity.com/paper/2311.00262/) (2023)
- [From Skepticism to Acceptance: Simulating the Attitude Dynamics Toward Fake News](https://mlsanity.com/paper/2403.09498/) (2024)
- [Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Tools](https://mlsanity.com/paper/2404.11891/) (2024)
- [LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning](https://mlsanity.com/paper/2407.02511/) (2024)
- [TravelAgent: An AI Assistant for Personalized Travel Planning](https://mlsanity.com/paper/2409.08069/) (2024)
- [Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning](https://mlsanity.com/paper/2405.20625/) (2024)
- [Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements](https://mlsanity.com/paper/2502.12904/) (2025)
- [Script-Based Dialog Policy Planning for LLM-Powered Conversational Agents: A Basic Architecture for an "AI Therapist"](https://mlsanity.com/paper/2412.15242/) (2024)

## Built on by (most cited first, 15 of 15)

- [Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models](https://mlsanity.com/paper/2508.10599/) (2025)
- [The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets](https://mlsanity.com/paper/2506.00073/) (2025)
- [AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor](https://mlsanity.com/paper/2601.05752/) (2026)
- [Predicting LLM Output Length via Entropy-Guided Representations](https://mlsanity.com/paper/2602.11812/) (2026)
- [When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms](https://mlsanity.com/paper/2511.06448/) (2025)
- [COMPKE: Complex Question Answering under Knowledge Editing](https://mlsanity.com/paper/2506.00829/) (2025)
- [Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models](https://mlsanity.com/paper/2508.02360/) (2025)
- [Can AI Truly Represent Your Voice in Deliberations? A Comprehensive Study of Large-Scale Opinion Aggregation with LLMs](https://mlsanity.com/paper/2510.05154/) (2025)
- [LLM Agents in Law: Taxonomy, Applications, and Challenges](https://mlsanity.com/paper/2601.06216/) (2026)
- [Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localization](https://mlsanity.com/paper/2509.07864/) (2025)
- [The Compositional Architecture of Regret in Large Language Models](https://mlsanity.com/paper/2506.15617/) (2025)
- [SHERLOCK: Towards Dynamic Knowledge Adaptation in LLM-enhanced E-commerce Risk Management](https://mlsanity.com/paper/2510.08948/) (2025)
- [HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debate](https://mlsanity.com/paper/2512.23717/) (2025)
- [Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs](https://mlsanity.com/paper/2606.09038/) (2026)
- [AISPA: User-Centric System Prompt Auditing for Large Language Model Applications](https://mlsanity.com/paper/2607.28617/) (2026)

## Abstract (arXiv)

The rise of Large Language Model-based Multi-Agent Planning has leveraged advanced frameworks to enable autonomous and collaborative task execution. Some systems rely on platforms like review sites and social media, which are prone to fraudulent information, such as fake reviews or misleading descriptions. This reliance poses risks, potentially causing financial losses and harming user experiences. To evaluate the risk of planning systems in real-world applications, we introduce WandaPlan, an evaluation environment mirroring real-world data and injected with deceptive content. We assess system performance across three fraud cases: Misinformation Fraud, Team-Coordinated Multi-Person Fraud, and Level-Escalating Multi-Round Fraud. We reveal significant weaknesses in existing frameworks that prioritize task efficiency over data authenticity. At the same time, we validate WandaPlan's generalizability, capable of assessing the risks of real-world open-source planning frameworks. To mitigate the risk of fraud, we propose integrating an anti-fraud agent, providing a solution for reliable planning.