Evaluating Steering Techniques using Human Similarity Judgments
Not on the map. Why?
Abstract (arXiv)
Current evaluations of Large Language Model (LLM) steering techniques focus on task-specific performance, overlooking how well steered representations align with human cognition. Using a well-established triadic similarity judgment task, we assessed steered LLMs on their ability to flexibly judge similarity between concepts based on size or kind, two central dimensions organizing human mental representations. We found that prompt-based steering methods outperformed other methods both in terms of steering accuracy and model-to-human alignment. We also found LLMs were biased towards `kind' similarity and struggled with `size' alignment. This evaluation approach, grounded in human cognition, adds further support to the efficacy of prompt-based steering and reveals privileged representational axes in LLMs prior to steering.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 63 references that are in the map. First posted on arXiv 25 May 2025.
Builds on
31 of 63 references.
References · 31
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
- 2023In-Context Learning Creates Task Vectorscited by 312
- 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
- 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
- 2018A mathematical theory of semantic development in deep neural networkscited by 186
19 more
- 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
- 2023Getting aligned on representational alignmentcited by 143
- 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
- 2023Why think step by step? Reasoning emerges from the locality of experiencecited by 103
- 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
- 2022Human alignment of neural network representationscited by 74
- 2023Aligning Large Language Models with Human Preferences through Representation Engineeringcited by 71
- 2024Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networkscited by 39
- 2023Improving neural network representations using human similarity judgmentscited by 38
- 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Modelscited by 34
- 2024Strong and weak alignment of large language models with human valuescited by 27
- 2024Finding Visual Task Vectorscited by 24
- 2024Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuningcited by 21
- 2023Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortexcited by 18
- 2022VICE: Variational Interpretable Concept Embeddingscited by 15
- 2024Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgettingcited by 15
- 2023Conceptual structure coheres in human cognition but not in large language modelscited by 14
- 2016Finite Sample Prediction and Recovery Bounds for Ordinal Embeddingcited by 12
- 2024What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Modelscited by 10
Built on by
Citing papers per year: 2025: 2.
2 papers.