mlsanityData to 14 Oct 2026

Efficient Universal Goal Hijacking with Semantics-guided Prompt Organization

arXiv 2405.14189 ↗First posted 23 May 2024cs.CL cs.CVcited by 16

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Universal goal hijacking is a kind of prompt injection attack that forces LLMs to return a target malicious response for arbitrary normal user prompts. The previous methods achieve high attack performance while being too cumbersome and time-consuming. Also, they have concentrated solely on optimization algorithms, overlooking the crucial role of the prompt. To this end, we propose a method called POUGH that incorporates an efficient optimization algorithm and two semantics-guided prompt organization strategies. Specifically, our method starts with a sampling strategy to select representative prompts from a candidate pool, followed by a ranking strategy that prioritizes them. Given the sequentially ranked prompts, our method employs an iterative optimization algorithm to generate a fixed suffix that can concatenate to arbitrary user prompts for universal goal hijacking. Experiments conducted on four popular LLMs and ten types of target responses verified the effectiveness.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 27 of its 51 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

27 of 51 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Automatic and Universal Prompt Injection Attacks against Large Language Models7 Mar
  2. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models21 Dec
  3. 2023Hijacking Large Language Models via Adversarial In-Context Learning16 Nov
  4. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game2 Nov
  5. 2023Prompt Injection attack against LLM-integrated Applications8 Jun
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  8. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  9. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
9 more
  1. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  2. 2023Are aligned neural networks adversarially aligned?cited by 315
  3. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
  4. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 172
  5. 2023Scaling Sentence Embeddings with Large Language Modelscited by 85
  6. 2022Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examplescited by 69
  7. 2024Boosting Jailbreak Attack with Momentumcited by 40
  8. 2019Amora: Black-box Adversarial Morphing Attackcited by 17
  9. 2021AdvFilter: Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 7.

16 papers.

Sort:

By area:

  1. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedbackcited by 72
  2. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 65
  3. 2024Large Language Model Safety: A Holistic Surveycited by 59
  4. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 43
  5. 2026SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agentscited by 25
  6. 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Modelscited by 19
  7. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  8. 2024GenTel-Safe: A Unified Benchmark and Shielding Framework for Defending Against Prompt Injection Attackscited by 7
  9. 2024DistillSeq: A Framework for Safety Alignment Testing in Large Language Models using Knowledge Distillationcited by 5
  10. 2025Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimizationcited by 5
  11. 2025ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generationcited by 5
  12. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Modelscited by 2
4 more
  1. 2026Reasoning Hijacking: The Fragility of Reasoning Alignment in Large Language Modelscited by 1
  2. 2026Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Headscited by 1
  3. 2026ECLIPSE: Self-Evolving Stealthy Prompt Injection Attack against Long-Horizon Agentic Systemscited by 0
  4. 2026Agentic Pressure: The Endogenous Entropy of Reliable Autonomycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond the Benchmark: Innovative Defenses Against Prompt Injection Attacks
  2. 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection
  3. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defenses
  4. 2024Universal and Context-Independent Triggers for Precise Control of LLM Outputs
  5. 2024HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Models
  6. 2025Prompt Optimization and Evaluation for LLM Automated Red Teaming
  7. 2025PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariance
  8. 2026CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimization