# Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoning

arXiv: https://arxiv.org/abs/2510.01032 · HTML: https://mlsanity.com/paper/2510.01032/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 31)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2401.17263/) (2024)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://mlsanity.com/paper/2410.12405/) (2024)
- [Thought Anchors: Which LLM Reasoning Steps Matter?](https://mlsanity.com/paper/2506.19143/) (2025)
- [Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning](https://mlsanity.com/paper/2507.00432/) (2025)
- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [Neurons in Large Language Models: Dead, N-gram, Positional](https://mlsanity.com/paper/2309.04827/) (2023)
- [Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspective](https://mlsanity.com/paper/2409.10053/) (2024)
- [Improving Reasoning Performance in Large Language Models via Representation Engineering](https://mlsanity.com/paper/2504.19483/) (2025)
- [Sparsing Law: Towards Large Language Models with Greater Activation Sparsity](https://mlsanity.com/paper/2411.02335/) (2024)
- [An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs](https://mlsanity.com/paper/2406.12288/) (2024)
- [The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs](https://mlsanity.com/paper/2502.04134/) (2025)
- [From Attention to Activation: Unravelling the Enigmas of Large Language Models](https://mlsanity.com/paper/2410.17174/) (2024)
- [LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers](https://mlsanity.com/paper/2502.15007/) (2025)
- [A Refined Analysis of Massive Activations in LLMs](https://mlsanity.com/paper/2503.22329/) (2025)
- [Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers](https://mlsanity.com/paper/2505.21024/) (2025)
- [Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient](https://mlsanity.com/paper/2412.18196/) (2024)
- [Defending Large Language Models Against Attacks With Residual Stream Activation Analysis](https://mlsanity.com/paper/2406.03230/) (2024)
- [Punctuation and Predicates in Language Models](https://mlsanity.com/paper/2508.14067/) (2025)
- [Punctuation Restoration Improves Structure Understanding Without Supervision](https://mlsanity.com/paper/2402.08382/) (2024)
- [Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applications](https://mlsanity.com/paper/2402.16367/) (2024)
- [InverseScope: Scalable Activation Inversion for Interpreting Large Language Models](https://mlsanity.com/paper/2506.07406/) (2025)

## Built on by (most cited first, 8 of 8)

- [Not All LLM Reasoning is Visible in the Chain-of-Thought](https://mlsanity.com/paper/2607.22925/) (2026)
- [What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis](https://mlsanity.com/paper/2606.20075/) (2026)
- [A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models](https://mlsanity.com/paper/2605.08504/) (2026)
- [The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context](https://mlsanity.com/paper/2607.12963/) (2026)
- [ART: Attention Replacement Technique to Improve Factuality in LLMs](https://mlsanity.com/paper/2604.06393/) (2026)
- [Nonsense Helps: Prompt Space Perturbation Broadens Reasoning Exploration](https://mlsanity.com/paper/2605.05566/) (2026)
- [Programs-of-Layers in LLMs through the Lens of Cortical Areas](https://mlsanity.com/paper/2609.31360/) (2026)
- [Post-training Quantization for Hybrid Iterative Generative Models](https://mlsanity.com/paper/2608.13932/) (2026)

## Abstract (arXiv)

Motivated by the puzzling observation that inserting long sequences of meaningless tokens before the query prompt can consistently enhance LLM reasoning performance, this work analyzes the underlying mechanism driving this phenomenon and based on these insights proposes a more principled method that allows for similar performance gains. First, we find that the improvements arise from a redistribution of activations in the LLM's MLP layers, where near zero activations become less frequent while large magnitude activations increase. This redistribution enhances the model's representational capacity by suppressing weak signals and promoting stronger, more informative ones. Building on this insight, we propose the Activation Redistribution Module (ARM), a lightweight inference-time technique that modifies activations directly without altering the input sequence. ARM adaptively identifies near-zero activations after the non-linear function and shifts them outward, implicitly reproducing the beneficial effects of meaningless tokens in a controlled manner. Extensive experiments across diverse benchmarks and model architectures clearly show that ARM consistently improves LLM performance on reasoning tasks while requiring only a few lines of simple code to implement. Our findings deliver both a clear mechanistic explanation for the unexpected benefits of meaningless tokens and a simple yet effective technique that harnesses activation redistribution to further improve LLM performance.