mlsanityData to 7 Oct 2026

Representation Engineering: A Top-Down Approach to AI Transparency

arXiv 2310.01405 ↗First posted 2 Oct 2023cs.LG cs.AI cs.CLcited by 1,239

Not on the map. Why?

Cited by 233 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. RepE places population-level representations, rather than neurons or circuits, at the center of analysis, equipping us with novel methods for monitoring and manipulating high-level cognitive phenomena in deep neural networks (DNNs). We provide baselines and an initial analysis of RepE techniques, showing that they offer simple yet effective solutions for improving our understanding and control of large language models. We showcase how these methods can provide traction on a wide range of safety-relevant problems, including honesty, harmlessness, power-seeking, and more, demonstrating the promise of top-down transparency research. We hope that this work catalyzes further exploration of RepE and fosters advancements in the transparency and safety of AI systems.

Key facts. Cited by 1,239 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Oct 2023.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 21; 2024: 245; 2025: 345; 2026: 628.

1,239 papers.

Sort:

By area:

  1. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  2. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  3. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  4. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  5. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  6. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  7. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  8. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  9. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
  10. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  11. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  12. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
38 more
  1. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  2. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  3. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  4. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  5. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  6. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  7. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  8. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  9. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  10. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  11. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  12. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
  13. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  14. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
  15. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  16. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  17. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  18. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  19. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  20. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  21. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  22. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 110
  23. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 109
  24. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  25. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  26. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  27. 2024Universal Neurons in GPT2 Language Modelscited by 100
  28. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  29. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  30. 2024Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedbackcited by 95
  31. 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 90
  32. 2024Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attackcited by 89
  33. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  34. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  35. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 86
  36. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  37. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  38. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82

50 most cited of 1,239 All 1,239 (JSON)