mlsanityData to 7 Oct 2026

SPML: A DSL for Defending Language Models Against Prompt Attacks

arXiv 2402.11755 ↗First posted 19 Feb 2024cs.LG cs.CL cs.CRcited by 33

Not on the map. Why?

Cited by 20 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have profoundly transformed natural language applications, with a growing reliance on instruction-based definitions for designing chatbots. However, post-deployment the chatbot definitions are fixed and are vulnerable to attacks by malicious users, emphasizing the need to prevent unethical applications and financial losses. Existing studies explore user prompts' impact on LLM-based chatbots, yet practical methods to contain attacks on application-specific chatbots remain unexplored. This paper presents System Prompt Meta Language (SPML), a domain-specific language for refining prompts and monitoring the inputs to the LLM-based chatbots. SPML actively checks attack prompts, ensuring user inputs align with chatbot definitions to prevent malicious execution on the LLM backbone, optimizing costs. It also streamlines chatbot definition crafting with programming language capabilities, overcoming natural language design challenges. Additionally, we introduce a groundbreaking benchmark with 1.8k system prompts and 20k user inputs, offering the inaugural language and benchmark for chatbot definition evaluation. Experiments across datasets demonstrate SPML's proficiency in understanding attacker prompts, surpassing models like GPT-4, GPT-3.5, and LLAMA. Our data and codes are publicly available at: https://prompt-compiler.github.io/SPML/.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 24 of its 49 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

24 of 49 references.

References · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  10. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  11. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  12. 2020Towards a Human-like Open-Domain Chatbotcited by 467
12 more
  1. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  2. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  3. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  4. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  5. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  6. 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 135
  7. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  8. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  9. 2023From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?cited by 53
  10. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  11. 2023Stealing the Decoding Algorithms of Language Modelscited by 19
  12. 2023Maatphor: Automated Variant Analysis for Prompt Injection Attackscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 18; 2026: 5.

33 papers.

Sort:

By area:

  1. 2025Defeating Prompt Injections by Designcited by 256
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  5. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  6. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  7. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  8. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  9. 2024Security of AI Agentscited by 24
  10. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  11. 2025Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitorscited by 21
  12. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
21 more
  1. 2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoringcited by 20
  2. 2025Attack and defense techniques in large language models: A survey and new perspectivescited by 13
  3. 2025An Embarrassingly Simple Defense Against LLM Abliteration Attackscited by 12
  4. 2024Diversity Helps Jailbreak Large Language Modelscited by 9
  5. 2025PromptPex: Automatic Test Generation for Language Model Promptscited by 9
  6. 2025What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMscited by 7
  7. 2026Willful Disobedience: Automatically Detecting Failures in Agentic Tracescited by 6
  8. 2024Lightweight Safety Classification Using Pruned Language Modelscited by 5
  9. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  10. 2024Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspectivecited by 3
  11. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 3
  12. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Momentscited by 2
  13. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  14. 2025Lumos: Let there be Language Model System Certificationcited by 1
  15. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  16. 2025Multilingual Collaborative Defense for Large Language Modelscited by 0
  17. 2025GenAI Security: Outsmarting the Bots with a Proactive Testing Frameworkcited by 0
  18. 2025Prompt Mechanisms in Medical Imaging: A Comprehensive Surveycited by 0
  19. 2026From Instruction to Output: The Role of Prompting in Modern NLGcited by 0
  20. 2026The Latent Diagnostic Taxonomy: A Framework for Constructing Classifiers and Diagnosing Their Decisions, Applied to Prompt Injection Detectioncited by 0
  21. 2026Ajar: Measuring Open Privilege in Agent Defensescited by 0