mlsanityData to 7 Oct 2026

When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications

arXiv 2605.10176 ↗First posted 11 May 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Natural language interfaces to structured databases are becoming increasingly common, largely due to advances in large language models (LLMs) that enable users to query data using conversational input rather than formal query languages such as SQL. While this paradigm significantly improves usability and accessibility, it introduces new security risks, particularly the amplification of SQL injection vulnerabilities through the prompt-to-SQL translation process. Malicious users can exploit these mechanisms by crafting adversarial prompts that manipulate model behavior and generate unsafe queries. In this work, we propose a multi-layered security framework designed to detect and mitigate LLM-mediated SQL injection attacks. The framework integrates a front-end security shield for prompt sanitization, an advanced threat detection model for behavioral and semantic anomaly identification, and a signature-based control layer for known attack patterns. We evaluate the proposed framework under diverse and realistic attack scenarios, including prompt injection, obfuscated SQL payloads, and context-manipulation attacks. To ensure robustness, we generate and curate a comprehensive benchmark dataset of adversarial prompts and assess performance across a fine-tuned LLM configuration. Experimental results demonstrate that the proposed approach achieves high detection accuracy while maintaining low false-positive rates, significantly improving the secure deployment of LLM-powered database applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 34 of its 68 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

34 of 68 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
  2. 2024Signed-Prompt: A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications15 Jan
  3. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defenses19 Oct
  4. 2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks16 Oct
  5. 2023Prompt Injection attack against LLM-integrated Applications8 Jun

Other references · 29

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2023A Survey on Multimodal Large Language Modelscited by 898
  6. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  8. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  9. 2024Tool Learning with Large Language Models: A Surveycited by 221
  10. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  11. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  12. 2022Few-shot training LLMs for project-specific code-summarizationcited by 128
17 more
  1. 2023Large Language Models in Law: A Surveycited by 128
  2. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilitiescited by 125
  3. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 124
  4. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  5. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  6. 2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
  7. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  8. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 34
  9. 2024Large Language Models in Cybersecurity: State-of-the-Artcited by 34
  10. 2024Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Techniquecited by 34
  11. 2023Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generationcited by 24
  12. 2023SQLPrompt: In-Context Text-to-SQL with Minimal Labeled Datacited by 16
  13. 2024Self-Evaluation as a Defense Against Adversarial Attacks on LLMscited by 13
  14. 2024Using LLM to select the right SQL Query from candidatescited by 12
  15. 2024A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Modelscited by 12
  16. 2025Dynamics of Adversarial Attacks on Large Language Model-Based Search Enginescited by 5
  17. 2024A Survey on Large Language Models from Concept to Implementationcited by 4

Built on by

Not cited yet.