T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in rapidly growing ecosystems such as the Model Context Protocol (MCP). To address this gap, we propose a trajectory-aware evolutionary search method, T-MAP, which leverages execution trajectories to guide the discovery of adversarial prompts. Our approach enables the automatic generation of attacks that not only bypass safety guardrails but also reliably realize harmful objectives through actual tool interactions. Empirical evaluations across diverse MCP environments demonstrate that T-MAP substantially outperforms baselines in attack realization rate (ARR) and remains effective against frontier models, including GPT-5.2, Gemini-3-Pro, Qwen3.5, and GLM-5, thereby revealing previously underexplored vulnerabilities in autonomous LLM agents. Code is available at https://github.com/pwnhyo/T-MAP.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 22 of its 30 references that are in the map. First posted on arXiv 21 Mar 2026.
Builds on
22 of 30 references.
Closest ancestors · 4
Same direction, newest first
- 2025SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning30 Oct
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
Other references · 18
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
- 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
6 more
- 2015Illuminating search spaces by mapping elitescited by 406
- 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
- 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 73
- 2025Adversarial Reasoning at Jailbreaking Timecited by 27
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Searchcited by 4
- 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teamingcited by 2
- 2026Automatic Hard Example Synthesis with Multi-Level Agentic Data Curationcited by 0
- 2026Governing Execution Risk in Agentic AI Systems: A Trajectory-Guided Framework for Red Teamingcited by 0
- 2026Breadcrumbing Search Agentscited by 0
- 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problemcited by 0
- 2026Rethinking Indirect Prompt Injection as a Test-Time Search Problem3 Sep 2026
- 2026Breadcrumbing Search Agents5 Aug 2026
- 2026Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation15 Jul 2026
- 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming13 Jul 2026
- 2026Governing Execution Risk in Agentic AI Systems: A Trajectory-Guided Framework for Red Teaming23 Jun 2026
- 2026What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search21 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025Automatic Red Teaming LLM-based Agents with Model Context Protocol Tools
- 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Models
- 2026TreeTeaming: Autonomous Red-Teaming of Vision-Language Models via Hierarchical Strategy Exploration
- 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs
- 2025Genesis: Evolving Attack Strategies for LLM Web Agent Red-Teaming
- 2026AgenticRed: Evolving Agentic Systems for Red-Teaming
- 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
- 2026RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution