mlsanityData to 7 Oct 2026

Extracting Latent Steering Vectors from Pretrained Language Models

arXiv 2205.05124 ↗First posted 10 May 2022cs.CL cs.AI cs.LGcited by 217

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Prior work on controllable text generation has focused on learning how to control language models through trainable decoding, smart-prompt design, or fine-tuning based on a desired objective. We hypothesize that the information needed to steer the model to generate a target sentence is already encoded within the model. Accordingly, we explore a different approach altogether: extracting latent vectors directly from pretrained language model decoders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly perfectly (> 99 BLEU) for English sentences from a variety of domains. We show that vector arithmetic can be used for unsupervised sentiment transfer on the Yelp sentiment benchmark, with performance comparable to models tailored to this task. We find that distances between steering vectors reflect sentence similarity when evaluated on a textual similarity benchmark (STS-B), outperforming pooled hidden states of models. Finally, we present an analysis of the intrinsic properties of the steering vectors. Taken together, our results suggest that frozen LMs can be effectively controlled through their latent steering space.

Key facts. Cited by 217 papers in the map (data through 7 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 10 May 2022.

Builds on

33 of 50 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2018Deep contextualized word representationscited by 2,285
  11. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  12. 2020A Primer in BERTology: What we know about how BERT workscited by 807
21 more
  1. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  2. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  3. 2015Generating Sentences from a Continuous Spacecited by 607
  4. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  5. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  6. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  7. 2017Toward Controlled Generation of Textcited by 271
  8. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  9. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  10. 2020Deep Learning for Text Style Transfer: A Surveycited by 161
  11. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  12. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 119
  13. 2020Residual Energy-Based Models for Text Generationcited by 95
  14. 2020A Probabilistic Formulation of Unsupervised Text Style Transfercited by 70
  15. 2021Tailor: Generating and Perturbing Text with Semantic Controlscited by 67
  16. 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
  17. 2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
  18. 2020Plug and Play Autoencoders for Conditional Text Generationcited by 26
  19. 2017Trainable Greedy Decoding for Neural Machine Translationcited by 23
  20. 2021Sentence Bottleneck Autoencoders from Transformer Language Modelscited by 19
  21. 2019Can Unconditional Language Models Recover Arbitrary Sentences?cited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 42; 2025: 63; 2026: 101.

217 papers.

Sort:

By area:

  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  2. 2022Editing Models with Task Arithmeticcited by 1,076
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  4. 2023Steering Language Models With Activation Engineeringcited by 824
  5. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  6. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  7. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  8. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  9. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  10. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  11. 2025MemOS: A Memory OS for AI Systemcited by 129
  12. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
38 more
  1. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  2. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  3. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristicscited by 94
  4. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  5. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  6. 2025Do Multilingual LLMs Think In English?cited by 75
  7. 2024Test-Time Model Adaptation with Only Forward Passescited by 74
  8. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 74
  9. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  10. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 69
  11. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 68
  12. 2024Aligning Large Language Models with Representation Editing: A Control Perspectivecited by 62
  13. 2024Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectorscited by 61
  14. 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 60
  15. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  16. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
  17. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 44
  18. 2025Learning without training: The implicit dynamics of in-context learningcited by 41
  19. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  20. 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
  21. 2025Understanding (Un)Reliability of Steering Vectors in Language Modelscited by 39
  22. 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
  23. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 32
  24. 2024Improving Uncertainty Quantification in Large Language Models via Semantic Embeddingscited by 30
  25. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  26. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  27. 2025One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMscited by 27
  28. 2024Do LLMs "know" internally when they follow instructions?cited by 25
  29. 2024Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Modelscited by 23
  30. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  31. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 22
  32. 2025Text Generation Beyond Discrete Token Samplingcited by 21
  33. 2024LiSeCo: Linear Semantic Control for Language Generationcited by 20
  34. 2023Understanding and Controlling a Maze-Solving Policy Networkcited by 19
  35. 2024Constructing Benchmarks and Interventions for Combating Hallucinations in LLMscited by 19
  36. 2024PaCE: Parsimonious Concept Engineering for Large Language Modelscited by 19
  37. 2025Steering Language Models with Weight Arithmeticcited by 19
  38. 2024A Causal Explainable Guardrails for Large Language Modelscited by 18

50 most cited of 217 All 217 (JSON)