mlsanityData to 7 Oct 2026

Representation Engineering for Large-Language Models: Survey and Research Challenges

arXiv 2502.17601 ↗First posted 24 Feb 2025cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Large-language models are capable of completing a variety of tasks, but remain unpredictable and intractable. Representation engineering seeks to resolve this problem through a new approach utilizing samples of contrasting inputs to detect and edit high-level representations of concepts such as honesty, harmfulness or power-seeking. We formalize the goals and methods of representation engineering to present a cohesive picture of work in this emerging discipline. We compare it with alternative approaches, such as mechanistic interpretability, prompt-engineering and fine-tuning. We outline risks such as performance decrease, compute time increases and steerability issues. We present a clear agenda for future research to build predictable, dynamic, safe and personalizable LLMs.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 158 of its 215 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

158 of 215 references.

References · 158

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2024The Llama 3 Herd of Modelscited by 14,192
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  11. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  12. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
38 more
  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  6. 2020Concept Bottleneck Modelscited by 1,022
  7. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  8. 2022Mass-Editing Memory in a Transformercited by 976
  9. 2022In-context Learning and Induction Headscited by 930
  10. 2022Toy Models of Superpositioncited by 928
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  13. 2016Universal adversarial perturbationscited by 774
  14. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  15. 2022A Survey on In-context Learningcited by 717
  16. 2022Transformers learn in-context by gradient descentcited by 714
  17. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  18. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  19. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  20. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  21. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  22. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  23. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  24. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  25. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  26. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  27. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  28. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  29. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  30. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  31. 2023Explainability for Large Language Models: A Surveycited by 331
  32. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  33. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  34. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  35. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  36. 2023In-Context Learning Creates Task Vectorscited by 302
  37. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  38. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240

50 most cited of 158 All 158 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 17.

18 papers.

Sort:

By area:

  1. 2026ODESteer: A Unified ODE-Based Steering Framework for LLM Alignmentcited by 15
  2. 2026Internalizing LLM Reasoning via Discovery and Replay of Latent Actionscited by 3
  3. 2026How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularitiescited by 3
  4. 2026Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventionscited by 3
  5. 2026PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifactscited by 3
  6. 2025Steering Information Utility in Key-Value Memory for Language Model Post-Trainingcited by 2
  7. 2026Composer Vector: Style-steering Symbolic Music Generation in a Latent Spacecited by 2
  8. 2026Activation Steering for Aligned Open-ended Generation without Sacrificing Coherencecited by 2
  9. 2026A Geometric Perspective on Composable Emotion Steering in Text-to-Speech Modelscited by 1
  10. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoningcited by 1
  11. 2026Mind the Gap: Pitfalls of LLM Alignment with Asian Public Opinioncited by 0
  12. 2026Structural Graph Probing of Vision-Language Modelscited by 0
6 more
  1. 2026Why MLLMs Struggle to Determine Object Orientationscited by 0
  2. 2026The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Modelscited by 0
  3. 2026Relational Intervention During Functional Collapse in Large Language Models: A Lexical-Statistical Ablation and a Structure x Register Factorialcited by 0
  4. 2026Emergent Alignmentcited by 0
  5. 2026FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilitiescited by 0
  6. 2026Position: It is Time to Virtualize Foundation Models with a Self-evolving Operating System Layercited by 0