mlsanityData to 7 Oct 2026

Involuntary In-Context Learning: Exploiting Few-Shot Pattern Completion to Bypass Safety Alignment in GPT-5.4

arXiv 2604.19461 ↗First posted 21 Apr 2026cs.CRcited by 2

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment in large language models relies on behavioral training that can be overridden when sufficiently strong in-context patterns compete with learned refusal behaviors. We introduce Involuntary In-Context Learning (IICL), an attack class that uses abstract operator framing with few-shot examples to force pattern completion that overrides safety training. Through 3479 probes across 10 OpenAI models, we identify the attack's effective components through a seven-experiment ablation study. Key findings: (1)~semantic operator naming achieves 100% bypass rate (50/50, p < 0.001); (2)~the attack requires abstract framing, since identical examples in direct question-and-answer format yield 0%; (3)~example ordering matters strongly (interleaved: 76%, harmful-first: 6%); (4)~temperature has no meaningful effect (46-56% across 0.0--1.0). On the HarmBench benchmark, IICL achieves 24.0% bypass [18.6%, 30.4%] against GPT-5.4 with detailed 619-word responses, compared to 0.0% for direct queries.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 21 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

16 of 21 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  8. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  9. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  10. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  11. 2024The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Modelscited by 20
  12. 2024SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chainscited by 16
1 more
  1. 2026ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attackcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Belief or Circuitry? Causal Evidence for In-Context Graph Learningcited by 0
  2. 2026Structural Jailbreaks Generalize but Do Not Compound: A cross-provider and multilingual study of Involuntary In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Context-Invariant Safety Alignment for Large Language Models
  2. 2026Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
  3. 2026DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models
  4. 2025Rethinking Deep Alignment Through The Lens Of Incomplete Learning
  5. 2026Safety in Batches? Understanding and Mitigating Safety Failures in Batch Prompting
  6. 2025Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs
  7. 2026Continual Safety Alignment via Gradient-Based Sample Selection
  8. 2026Breaking Safety at the Token Boundary: How BPE Tokenization Creates Exploitable Gaps in LLM Alignment