Evaluating Jailbreaking Vulnerabilities in LLMs Deployed as Assistants for Smart Grid Operations: A Benchmark Against NERC Standards
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
The deployment of Large Language Models (LLMs) as assistants in electric grid operations promises to streamline compliance and decision-making but exposes new vulnerabilities to prompt-based adversarial attacks. This paper evaluates the risk of jailbreaking LLMs, i.e., circumventing safety alignments to produce outputs violating regulatory standards, assuming threats from authorized users, such as operators, who craft malicious prompts to elicit non-compliant guidance. Three state-of-the-art LLMs (OpenAI's GPT-4o mini, Google's Gemini 2.0 Flash-Lite, and Anthropic's Claude 3.5 Haiku) were tested against Baseline, BitBypass, and DeepInception jailbreaking methods across scenarios derived from nine NERC Reliability Standards (EOP, TOP, and CIP). In the initial broad experiment, the overall Attack Success Rate (ASR) was 33.1%, with DeepInception proving most effective at 63.17% ASR. Claude 3.5 Haiku exhibited complete resistance (0% ASR), while Gemini 2.0 Flash-Lite was most vulnerable (55.04% ASR) and GPT-4o mini moderately susceptible (44.34% ASR). A follow-up experiment refining malicious wording in Baseline and BitBypass attacks yielded a 30.6% ASR, confirming that subtle prompt adjustments can enhance simpler methods' efficacy.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 9 of its 31 references that are in the map. First posted on arXiv 25 Apr 2026.
Builds on
9 of 31 references.
Closest ancestors · 3
Same direction, newest first
- 2025BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage3 Jun
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
- 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
Other references · 6
- 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
- 2023ChatGPT and Other Large Language Models for Cybersecurity of Smart Grid Applicationscited by 13
- 2024Towards Explainable Network Intrusion Detection using Large Language Modelscited by 12
- 2025PowerGraph-LLM: Novel Power Grid Graph Embedding and Optimization with Large Language Modelscited by 6
- 2025Large Language Model-Based Framework for Explainable Cyberattack Detection in Automatic Generation Control Systemscited by 1
- 2026Large Language Models for Detecting Cyberattacks on Smart Grid Protective Relayscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Analysis of LLMs Against Prompt Injection and Jailbreak Attacks
- 2026The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
- 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
- 2025SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
- 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
- 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
- 2025Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs
- 2025Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy