Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
Not on the map. Why?
Abstract (arXiv)
Activation steering controls model behavior by editing internal activations at inference time. We study its input-side dual: optimizing a fluent prompt so that a chosen internal latent is driven toward zero, with no inference-time model access. Our target is an "evaluation-awareness" latent-linearly readable and steerable in recent work-whose control would threaten the validity of safety evaluations if models behave differently when they detect being tested. Adapting Fluent Dreaming / EPO with a negated feature term (GCG-style token optimization plus a self-cross-entropy fluency regularizer, swept over a fluency weight), we suppress the latent under five target constructions-a CAA direction, a subspace norm, an SAE feature, a single MLP neuron, and a behavioral logit-on Llama-3.2-3B and Llama-3.1-8B. The latent is robustly suppressible (z≈-7), and a causally-validated Llama Scope SAE feature can be fully and selectively turned off. But our controls tell a cautionary story about the CAA direction: a placebo random direction is suppressed just as hard and shifts behavior just as far, and when we hold a real eval passage in context and optimize only a prefix, suppressing the eval-direction fails to reduce-and slightly increases-the model's behavioral eval judgment. Activation-readability, in short, is not behavioral controllability. We further find that a single MLP neuron is eval-correlated but not causal at both scales, and that scanning the real Pile yields a natural-text baseline competitive with the optimizer for the internal direction. A positive control validates our erasure detector, bounding an erasure-vs-rotation question earlier left open.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 28 Jul 2026.
Builds on
22 of 32 references.
References · 22
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
- 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
- 2023Detecting Language Model Attacks with Perplexitycited by 411
10 more
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
- 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
- 2023LEACE: Perfect linear concept erasure in closed formcited by 254
- 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 133
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
- 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
- 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
- 2025Steering Evaluation-Aware Language Models to Act Like They Are Deployedcited by 30
- 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
- 2023Investigating the Encoding of Words in BERT's Neurons using Feature Textualizationcited by 3
Built on by
Not cited yet.