From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?
On the map: LLM safety and security → Prompt injection and agent security
Abstract (arXiv)
Large Language Models (LLMs) have found widespread applications in various domains, including web applications, where they facilitate human interaction via chatbots with natural language interfaces. Internally, aided by an LLM-integration middleware such as Langchain, user prompts are translated into SQL queries used by the LLM to provide meaningful responses to users. However, unsanitized user prompts can lead to SQL injection attacks, potentially compromising the security of the database. Despite the growing interest in prompt injection vulnerabilities targeting LLMs, the specific risks of generating SQL injection attacks through prompt injections have not been extensively studied. In this paper, we present a comprehensive examination of prompt-to-SQL (P2SQL) injections targeting web applications based on the Langchain framework. Using Langchain as our case study, we characterize P2SQL injections, exploring their variants and impact on application security through multiple concrete examples. Furthermore, we evaluate 7 state-of-the-art LLMs, demonstrating the pervasiveness of P2SQL attacks across language models. Our findings indicate that LLM-integrated applications based on Langchain are highly susceptible to P2SQL injection attacks, warranting the adoption of robust defenses. To counter these attacks, we propose four effective defense techniques that can be integrated as extensions to the Langchain framework. We validate the defenses through an experimental evaluation with a real-world use case application.
Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. First posted on arXiv 3 Aug 2023.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2023: 2; 2024: 23; 2025: 23; 2026: 5.
53 papers.
Sort:
By area:
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
- 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
- 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
- 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakagecited by 146
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systemscited by 118
- 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 112
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 106
38 more
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
- 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systemscited by 74
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
- 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
- 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Modelscited by 36
- 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
- 2024SPML: A DSL for Defending Language Models Against Prompt Attackscited by 33
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
- 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
- 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applicationscited by 17
- 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
- 2024Attacks on Third-Party APIs of Large Language Modelscited by 14
- 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 14
- 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systemscited by 12
- 2025Prompt Injection 2.0: Hybrid AI Threatscited by 12
- 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
- 2025LLM Applications: Current Paradigms and the Next Frontiercited by 8
- 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agentscited by 8
- 2025RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelinescited by 5
- 2025Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domaincited by 5
- 2024Towards Action Hijacking of Large Language Model-based Agentcited by 4
- 2025Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Modelscited by 4
- 2024On the Feasibility of Fully AI-automated Vishing Attackscited by 3
- 2025Web Intellectual Property at Risk: Preventing Unauthorized Real-Time Retrieval by Large Language Modelscited by 3
- 2026What If Prompt Injection Never Left? Rethinking Agent Security through Cross-Session Stored Prompt Injectioncited by 3
- 2025Effective and Efficient Jailbreaks of Black-Box LLMs with Cross-Behavior Attackscited by 2
- 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyondcited by 1
- 2025Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systemscited by 1
- 2025Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agentscited by 1
- 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
- 2025Phi: Preference Hijacking in Multi-modal Large Language Models at Inference Timecited by 1
- 2025AudAgent: Automated Auditing of Privacy Policy Compliance in AI Agentscited by 1
- 2025Detecting Prompt Injection Attacks Against Application Using Classifierscited by 1
- 2025Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breakingcited by 1
- 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
- 2024Fortifying Ethical Boundaries in AI: Advanced Strategies for Enhancing Security in Large Language Modelscited by 0
50 most cited of 53 All 53 (JSON)
- 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions14 Jul 2026
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents25 Jun 2026
- 2026What If Prompt Injection Never Left? Rethinking Agent Security through Cross-Session Stored Prompt Injection3 Jun 2026
- 2026LogicPoison: Logical Attacks on Graph Retrieval-Augmented Generation3 Apr 2026
- 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents13 Mar 2026
- 2025Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breaking24 Dec 2025
- 2025Detecting Prompt Injection Attacks Against Application Using Classifiers14 Dec 2025
- 2025AudAgent: Automated Auditing of Privacy Policy Compliance in AI Agents3 Nov 2025
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges27 Oct 2025
- 2025Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models9 Oct 2025
- 2025The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking23 Sep 2025
- 2025Phi: Preference Hijacking in Multi-modal Large Language Models at Inference Time15 Sep 2025
38 more
- 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems12 Sep 2025
- 2025Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain4 Sep 2025
- 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use6 Aug 2025
- 2025Prompt Injection 2.0: Hybrid AI Threats17 Jul 2025
- 2025Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agents3 Jul 2025
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows29 Jun 2025
- 2025Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems23 Jun 2025
- 2025RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelines3 Jun 2025
- 2025Web Intellectual Property at Risk: Preventing Unauthorized Real-Time Retrieval by Large Language Models19 May 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar 2025
- 2025Effective and Efficient Jailbreaks of Black-Box LLMs with Cross-Behavior Attacks12 Mar 2025
- 2025LLM Applications: Current Paradigms and the Next Frontier6 Mar 2025
- 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation5 Feb 2025
- 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety2 Feb 2025
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmap13 Jan 2025
- 2024Towards Action Hijacking of Large Language Model-based Agent14 Dec 2024
- 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models30 Oct 2024
- 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond9 Oct 2024
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct 2024
- 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI23 Sep 2024
- 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs23 Sep 2024
- 2024On the Feasibility of Fully AI-automated Vishing Attacks20 Sep 2024
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage17 Sep 2024
- 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems3 Sep 2024
- 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications13 Jun 2024
- 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents5 Jun 2024
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways4 Jun 2024
- 2024AI Risk Management Should Incorporate Both Safety and Security29 May 2024
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Review6 May 2024
- 2024Attacks on Third-Party APIs of Large Language Models24 Apr 2024
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar 2024
- 2024Automatic and Universal Prompt Injection Attacks against Large Language Models7 Mar 2024
- 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems28 Feb 2024
- 2024SPML: A DSL for Defending Language Models Against Prompt Attacks19 Feb 2024
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb 2024
- 2024Security and Privacy Challenges of Large Language Models: A Survey30 Jan 2024
- 2024Fortifying Ethical Boundaries in AI: Advanced Strategies for Enhancing Security in Large Language Models27 Jan 2024
50 newest of 53 All 53 (JSON)
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
- 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakagecited by 146
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systemscited by 118
- 2024BadAgent: Inserting and Activating Backdoor Attacks in LLM Agentscited by 106
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
- 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
- 2025LLM Applications: Current Paradigms and the Next Frontiercited by 8
- 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agentscited by 8
- 2024Towards Action Hijacking of Large Language Model-based Agentcited by 4
- 2025Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agentscited by 1
12 most cited of 15 All 15 (JSON)
- 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions14 Jul 2026
- 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents25 Jun 2026
- 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents13 Mar 2026
- 2025AudAgent: Automated Auditing of Privacy Policy Compliance in AI Agents3 Nov 2025
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges27 Oct 2025
- 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use6 Aug 2025
- 2025Your Mailbox Is Mine: Prompt Injection Attacks Against Real-World LLM Email Agents3 Jul 2025
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows29 Jun 2025
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar 2025
- 2025LLM Applications: Current Paradigms and the Next Frontier6 Mar 2025
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmap13 Jan 2025
- 2024Towards Action Hijacking of Large Language Model-based Agent14 Dec 2024
12 newest of 15 All 15 (JSON)
- 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
- 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
- 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 112
- 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Modelscited by 36
- 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMscited by 17
- 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 14
- 2025Prompt Injection 2.0: Hybrid AI Threatscited by 12
- 2025Effective and Efficient Jailbreaks of Black-Box LLMs with Cross-Behavior Attackscited by 2
- 2025Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systemscited by 1
- 2025Detecting Prompt Injection Attacks Against Application Using Classifierscited by 1
- 2025Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breakingcited by 1
- 2025Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breaking24 Dec 2025
- 2025Detecting Prompt Injection Attacks Against Application Using Classifiers14 Dec 2025
- 2025Prompt Injection 2.0: Hybrid AI Threats17 Jul 2025
- 2025Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems23 Jun 2025
- 2025Effective and Efficient Jailbreaks of Black-Box LLMs with Cross-Behavior Attacks12 Mar 2025
- 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models30 Oct 2024
- 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs3 Oct 2024
- 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI23 Sep 2024
- 2024PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs23 Sep 2024
- 2024Automatic and Universal Prompt Injection Attacks against Large Language Models7 Mar 2024
- 2024Security and Privacy Challenges of Large Language Models: A Survey30 Jan 2024
- 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks16 Oct 2023
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2025Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Modelscited by 4
- 2026What If Prompt Injection Never Left? Rethinking Agent Security through Cross-Session Stored Prompt Injectioncited by 3
- 2026LogicPoison: Logical Attacks on Graph Retrieval-Augmented Generationcited by 0
- 2026What If Prompt Injection Never Left? Rethinking Agent Security through Cross-Session Stored Prompt Injection3 Jun 2026
- 2026LogicPoison: Logical Attacks on Graph Retrieval-Augmented Generation3 Apr 2026
- 2025Role-Conditioned Refusals: Evaluating Access Control Reasoning in Large Language Models9 Oct 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
- 2025RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelinescited by 5
- 2025Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domaincited by 5
- 2025Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domain4 Sep 2025
- 2025RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelines3 Jun 2025
- 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation5 Feb 2025
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb 2024
- 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
- 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systemscited by 74
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
- 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
- 2024SPML: A DSL for Defending Language Models Against Prompt Attackscited by 33
- 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
- 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applicationscited by 17
- 2024Attacks on Third-Party APIs of Large Language Modelscited by 14
- 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systemscited by 12
- 2024On the Feasibility of Fully AI-automated Vishing Attackscited by 3
- 2025Web Intellectual Property at Risk: Preventing Unauthorized Real-Time Retrieval by Large Language Modelscited by 3
12 most cited of 16 All 16 (JSON)
- 2025The Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking23 Sep 2025
- 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems12 Sep 2025
- 2025Web Intellectual Property at Risk: Preventing Unauthorized Real-Time Retrieval by Large Language Models19 May 2025
- 2024Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond9 Oct 2024
- 2024On the Feasibility of Fully AI-automated Vishing Attacks20 Sep 2024
- 2024SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems3 Sep 2024
- 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applications13 Jun 2024
- 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways4 Jun 2024
- 2024AI Risk Management Should Incorporate Both Safety and Security29 May 2024
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Review6 May 2024
- 2024Attacks on Third-Party APIs of Large Language Models24 Apr 2024
- 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems28 Feb 2024
12 newest of 16 All 16 (JSON)