mlsanityData to 7 Oct 2026

Steering Awareness: Detecting Activation Steering from Within

arXiv 2511.21399 ↗First posted 26 Nov 2025cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Activation steering -- adding a vector to a model's residual stream to modify its behavior -- is widely used in safety evaluations as if the model cannot detect the intervention. We test this assumption, introducing steering awareness: a model's ability to infer, during its own forward pass, that a steering vector was injected and what concept it encodes. After fine-tuning, seven instruction-tuned models develop strong steering awareness on held-out concepts; the best reaches 95.5% detection, 71.2% concept identification, and zero false positives on clean inputs. This generalizes to unseen steering vector construction methods when their directions have high cosine similarity to the training distribution but not otherwise, indicating a geometric detector rather than a generic anomaly detector. Surprisingly, detection does not confer resistance; on both factual and safety benchmarks, detection-trained models are consistently more susceptible to steering than their base counterparts. Mechanistically, steering awareness arises not from a localized circuit, but from a distributed transformation that progressively rotates diverse injected vectors into a shared detection direction. Activation steering should therefore not be considered an invisible intervention in safety evaluations.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

23 of 39 references.

References · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  10. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  11. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  12. 2023Steering Language Models With Activation Engineeringcited by 824
11 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  2. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
  3. 2025Persona Vectors: Monitoring and Controlling Character Traits in Language Modelscited by 303
  4. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 222
  5. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  6. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
  7. 2024Looking Inward: Language Models Can Learn About Themselves by Introspectioncited by 105
  8. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  9. 2025Steering Evaluation-Aware Language Models to Act Like They Are Deployedcited by 30
  10. 2025Privileged Self-Access Matters for Introspection in AIcited by 20
  11. 2025Prefill-level Jailbreak: A Black-Box Risk Analysis of Large Language Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.