# Language models recognize dropout and Gaussian noise applied to their activations

arXiv: https://arxiv.org/abs/2604.17465 · HTML: https://mlsanity.com/paper/2604.17465/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Reducing Transformer Depth on Demand with Structured Dropout](https://mlsanity.com/paper/1909.11556/) (2019)
- [LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding](https://mlsanity.com/paper/2404.16710/) (2024)
- [Emergent Introspective Awareness in Large Language Models](https://mlsanity.com/paper/2601.01828/) (2026)
- [Explicit Regularisation in Gaussian Noise Injections](https://mlsanity.com/paper/2007.07368/) (2020)
- [Scheduled DropHead: A Regularization Method for Transformer Models](https://mlsanity.com/paper/2004.13342/) (2020)
- [Does It Make Sense to Speak of Introspection in Large Language Models?](https://mlsanity.com/paper/2506.05068/) (2025)
- [Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models](https://mlsanity.com/paper/2412.01784/) (2024)
- [International AI Safety Report 2026](https://mlsanity.com/paper/2602.21012/) (2026)
- [DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks](https://mlsanity.com/paper/1907.11065/) (2019)
- [Latent Introspection: Models Can Detect Prior Concept Injections](https://mlsanity.com/paper/2602.20031/) (2026)
- [Enhancing Hallucination Detection through Noise Injection](https://mlsanity.com/paper/2502.03799/) (2025)
- [Drop Dropout on Single-Epoch Language Model Pretraining](https://mlsanity.com/paper/2505.24788/) (2025)

## Built on by (most cited first, 5 of 5)

- [Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment](https://mlsanity.com/paper/2606.23700/) (2026)
- [Safety from Honesty in a Disinterested AI Predictor](https://mlsanity.com/paper/2606.29657/) (2026)
- [Evaluation Awareness in Language Models: Representation, Verbalization, and Control](https://mlsanity.com/paper/2608.21766/) (2026)
- [Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference](https://mlsanity.com/paper/2609.34049/) (2026)
- [Cross-Model In-Context Neurofeedback Reveals (Limited) Activation Control in LLMs](https://mlsanity.com/paper/2610.20179/) (2026)

## Abstract (arXiv)

We provide evidence that language models can detect, localize and, to a degree, verbalize the difference between a-semantic perturbations applied to their activations. More precisely, we either (a) apply dropout or (b) add Gaussian noise to them, at a target sentence. We then ask a multiple-choice question such as "Which of the previous sentences was perturbed?" or "Which perturbation was applied?". We test eight models from the Gemma, Llama, Mistral, Olmo, Qwen and Seed families, with sizes between 8B and 36B, all of which can easily detect and localize the perturbations, often with perfect accuracy. To investigate the mechanisms, we find that Olmo models develop this capacity during pre-training, and we rule out the hypothesis that perturbed sentences simply have lower log-likelihood. Notably, the zero-shot accuracy of Qwen3-32B and Qwen3.8-27B in identifying which perturbation was applied correlates with the perturbation strength. While this task proves hard for the remaining models, most of them can learn, when taught in context, to distinguish between dropout and Gaussian noise. Interestingly, Qwen3-32B and Qwen3.8-27B learn worse when supervised with wrong labels, again suggesting a prior towards the correct ones. Because dropout is used as a training-regularization technique, while Gaussian noise is sometimes added during inference, we discuss the possibility of a data-agnostic "training awareness" signal and the implications for AI safety.