mlsanityData to 7 Oct 2026

A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models

arXiv 2607.05615 ↗First posted 6 Jul 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability p, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only 50% of the tokens recovers most of the dense-steering effect while preserving fluency, and steering as few as 30% surpasses prompt-based control. The optimal steering magnitude scales inversely with the intervention ratio, revealing that SAE-mediated control is rate-limited: the behavioral outcome depends on cumulative signal dosage across a sequence.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 32 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

26 of 32 references.

References · 26

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  10. 2023Steering Language Models With Activation Engineeringcited by 824
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  12. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
14 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  2. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  3. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  4. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  5. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  6. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  7. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  8. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  9. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  10. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 218
  11. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  12. 2025Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decodingcited by 7
  13. 2025Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data?cited by 5
  14. 2026Inference-time Alignment via Sparse Junction Steeringcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models
  2. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering
  3. 2026Reliable Control-Point Selection for Steering Reasoning in Large Language Models
  4. 2026Behavioral Steering in a 35B MoE Language Model via SAE-Decoded Probe Vectors: One Agency Axis, Not Five Traits
  5. 2024Controlling Large Language Model Agents with Entropic Activation Steering
  6. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms
  7. 2026CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
  8. 2026MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptation