# Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security

arXiv: https://arxiv.org/abs/2605.27823 · HTML: https://mlsanity.com/paper/2605.27823/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 21 of its 68 references that are in the map.
- First posted on arXiv 27 May 2026.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 21 of 21)

- [Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2401.17263/) (2024)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [Rule Based Rewards for Language Model Safety](https://mlsanity.com/paper/2411.01111/) (2024)
- [Adv-watermark: A Novel Watermark Perturbation for Adversarial Examples](https://mlsanity.com/paper/2008.01919/) (2020)
- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignment](https://mlsanity.com/paper/2505.21494/) (2025)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Hierarchical Local-Global Transformer for Temporal Sentence Grounding](https://mlsanity.com/paper/2208.14882/) (2022)
- [Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory](https://mlsanity.com/paper/2403.12445/) (2024)
- [Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Grounding](https://mlsanity.com/paper/2203.02966/) (2022)
- [Unbalanced Incomplete Multi-view Clustering via the Scheme of View Evolution: Weak Views are Meat; Strong Views do Eat](https://mlsanity.com/paper/2011.10254/) (2020)
- [ANIMC: A Soft Framework for Auto-weighted Noisy and Incomplete Multi-view Clustering](https://mlsanity.com/paper/2011.10331/) (2020)
- [Hypotheses Tree Building for One-Shot Temporal Sentence Localization](https://mlsanity.com/paper/2301.01871/) (2023)
- [Double Self-weighted Multi-view Clustering via Adaptive View Fusion](https://mlsanity.com/paper/2011.10396/) (2020)
- [Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack](https://mlsanity.com/paper/2411.02669/) (2024)
- [Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network](https://mlsanity.com/paper/2412.15678/) (2024)
- [Taylor Series-Inspired Local Structure Fitting Network for Few-shot Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2504.02454/) (2025)
- [To Align or Not to Align: Strategic Multimodal Representation Alignment for Optimal Performance](https://mlsanity.com/paper/2511.12121/) (2025)
- [Prototype-Driven Structure Synergy Network for Remote Sensing Images Segmentation](https://mlsanity.com/paper/2508.04022/) (2025)
- [Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimization](https://mlsanity.com/paper/2605.26501/) (2026)
- [RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Training](https://mlsanity.com/paper/2312.04032/) (2023)

## Built on by (most cited first, 8 of 8)

- [Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language](https://mlsanity.com/paper/2605.29812/) (2026)
- [Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language](https://mlsanity.com/paper/2605.29793/) (2026)
- [CogniVerse: Revolutionizing Multi-Modal Retrieval-Augmented Generation with Cognitive Reflection and Geometric Reasoning](https://mlsanity.com/paper/2605.29602/) (2026)
- [Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness](https://mlsanity.com/paper/2605.30745/) (2026)
- [SLAP: The Semantic Least Action Principle for Variational Video-Language Modeling](https://mlsanity.com/paper/2605.30750/) (2026)
- [Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation](https://mlsanity.com/paper/2606.01565/) (2026)
- [Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrieval](https://mlsanity.com/paper/2606.01615/) (2026)
- [Let Prompts Bridge Defense Knowledge: Transferable Graph Purification via Vulnerability-Aware GPL](https://mlsanity.com/paper/2608.29054/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly vulnerable to adversarial prompts that exploit semantic ambiguities to bypass safety mechanisms, resulting in harmful or inappropriate outputs. Such attacks, including jailbreaking and prompt injection, pose significant risks to the integrity and availability of LLMs in security-critical applications. This paper proposes the Adversarial Prompt Disentanglement (APD) framework, a novel defense mechanism that proactively identifies and neutralizes malicious components in input prompts before they are processed by the LLM. The APD framework integrates three key innovations: (1) a mutual information-based semantic decomposition method to isolate adversarial and benign prompt components, ensuring statistical independence; (2) a graph-based intent classification approach that leverages spectral analysis to detect malicious patterns in prompt semantics; and (3) a lightweight transformer-based classifier trained on real-world datasets of toxic and jailbreaking prompts, enabling efficient and accurate adversarial intent detection. Evaluated on diverse datasets containing adversarial prompts, APD demonstrates superior robustness, reducing harmful output generation by over 85% while maintaining negligible impact on model performance. The framework's computational efficiency supports real-time deployment, making it a practical solution for securing LLMs. Our work addresses critical challenges in machine learning security on novel attacks and integrity methods for ML systems, and offers a scalable, ethically grounded defense against prompt-based adversarial threats.