mlsanityData to 7 Oct 2026

Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning

arXiv 2407.03391 ↗First posted 3 Jul 2024cs.CR cs.AI cs.CLcited by 3

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Prompt injection (both direct and indirect) and jailbreaking are now recognized as significant issues for large language models (LLMs), particularly due to their potential for harm in application-integrated contexts. This extended abstract explores a novel approach to protecting LLMs from such attacks, termed "soft begging." This method involves training soft prompts to counteract the effects of corrupted prompts on the LLM's output. We provide an overview of prompt injections and jailbreaking, introduce the theoretical basis of the "soft begging" technique, and discuss an evaluation of its effectiveness.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 13 of its 16 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

13 of 16 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Detecting Language Model Attacks with Perplexity27 Aug
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 11

  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  5. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  6. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  7. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  8. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  9. 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 171
  10. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  11. 2023Hijacking Large Language Models via Adversarial In-Context Learningcited by 41

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025CASE-Bench: Context-Aware SafEty Benchmark for Large Language Modelscited by 17
  2. 2025Detection Method for Prompt Injection by Integrating Pre-trained Model and Heuristic Feature Engineeringcited by 4
  3. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0