mlsanityData to 7 Oct 2026

Test-Time Safety Alignment

arXiv 2604.26167 ↗First posted 28 Apr 2026cs.CL cs.AI cs.LGcited by 2

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Recent work has shown that a model's input word embeddings can serve as effective control variables for steering its behavior toward outputs that satisfy desired properties. However, this has only been demonstrated for pretrained text-completion models on the relatively simple objective of reducing surface-level profanity in short continuations. A natural and practically important question is how well input embeddings can control aligned models, which produce an imbalanced bimodal refuse-or-comply output distribution rather than the smooth distribution characteristic of open-ended generation. We explore this in the context of safety, showing that input word embeddings can be optimized in a sub-lexical manner to minimize the semantic harmfulness of aligned model responses. Our approach uses zeroth-order gradient estimation of a black-box text-moderation API with respect to the input embeddings, and then applies gradient descent on these embeddings to minimize the harmfulness of the generated text. Experiments show that the proposed method can neutralize every safety-flagged response on standard safety benchmarks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 36 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

31 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender13 Apr
  2. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  3. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance20 Jan
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  10. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  11. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  12. 2023Detecting Language Model Attacks with Perplexitycited by 411
15 more
  1. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  2. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  3. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  4. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  5. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  6. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  7. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  8. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 87
  9. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  10. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 63
  11. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 46
  12. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  13. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representationscited by 14
  14. 2025Smoothed Embeddings for Robust Language Modelscited by 8
  15. 2026Test-Time Detoxification without Training or Learning Anythingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Test-Time Training Undermines Safety Guardrailscited by 0
  2. 2026Self-Improving In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shift
  2. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
  3. 2024Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations
  4. 2026Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
  5. 2025Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
  6. 2025Safety Alignment Depth in Large Language Models: A Markov Chain Perspective
  7. 2025SEA: Low-Resource Safety Alignment for Multimodal Large Language Models via Synthetic Embeddings
  8. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection