mlsanityData to 7 Oct 2026

ReFT: Representation Finetuning for Language Models

arXiv 2404.03592 ↗First posted 4 Apr 2024cs.CL cs.AI cs.LGcited by 186

Not on the map. Why?

Abstract (arXiv)

Parameter-efficient finetuning (PEFT) methods seek to adapt large neural models via updates to a small number of weights. However, much prior interpretability work has shown that representations encode rich semantic information, suggesting that editing representations might be a more powerful alternative. We pursue this hypothesis by developing a family of Representation Finetuning (ReFT) methods. ReFT methods operate on a frozen base model and learn task-specific interventions on hidden representations. We define a strong instance of the ReFT family, Low-rank Linear Subspace ReFT (LoReFT), and we identify an ablation of this method that trades some performance for increased efficiency. Both are drop-in replacements for existing PEFTs and learn interventions that are 15x--65x more parameter-efficient than LoRA. We showcase LoReFT on eight commonsense reasoning tasks, four arithmetic reasoning tasks, instruction-tuning, and GLUE. In all these evaluations, our ReFTs deliver the best balance of efficiency and performance, and almost always outperform state-of-the-art PEFTs. We release a generic ReFT training library publicly at https://github.com/stanfordnlp/pyreft.

Key facts. Cited by 186 papers in the map (data through 7 Oct 2026). Builds on 79 of its 109 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

79 of 109 references.

References · 79

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
38 more
  1. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  3. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  4. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2022Toy Models of Superpositioncited by 928
  11. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  13. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  14. 2023Steering Language Models With Activation Engineeringcited by 824
  15. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  16. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  17. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  18. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  19. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  20. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  21. 2021Causal Abstractions of Neural Networkscited by 401
  22. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 359
  23. 2016Solving General Arithmetic Word Problemscited by 353
  24. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  25. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  26. 2023VeRA: Vector-based Random Matrix Adaptationcited by 294
  27. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  28. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  29. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  30. 2015Semi-supervised Sequence Learningcited by 198
  31. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  32. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  33. 2022Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Modelscited by 187
  34. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  35. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 155
  36. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  37. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  38. 2023Parameter-Efficient Orthogonal Finetuning via Butterfly Factorizationcited by 112

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 37; 2025: 69; 2026: 80.

186 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  3. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  4. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  5. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  6. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 124
  7. 2024MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuningcited by 89
  8. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  9. 2024Aligning Large Language Models with Representation Editing: A Control Perspectivecited by 62
  10. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  11. 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
  12. 2024LoFiT: Localized Fine-tuning on LLM Representationscited by 56
38 more
  1. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
  2. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  3. 2024A safety realignment framework via subspace-oriented model fusion for large language modelscited by 40
  4. 2025A Unified Understanding and Evaluation of Steering Methodscited by 40
  5. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 35
  6. 2025FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Expertscited by 34
  7. 2025LLM Pretraining with Continuous Conceptscited by 33
  8. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  9. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  10. 2025Representation Bending for Large Language Model Safetycited by 28
  11. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 27
  12. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 26
  13. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  14. 2024Locking Down the Finetuned LLMs Safetycited by 22
  15. 2025Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMscited by 22
  16. 2025Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuningcited by 22
  17. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  18. 2025Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuningcited by 21
  19. 2025Prompt-based Adaptation in Large-scale Vision Models: A Surveycited by 21
  20. 2024LiSeCo: Linear Semantic Control for Language Generationcited by 20
  21. 2024RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuningcited by 20
  22. 2025Localized Cultural Knowledge is Conserved and Controllable in Large Language Modelscited by 19
  23. 2024Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuningcited by 18
  24. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  25. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 17
  26. 2025Improved Representation Steering for Language Modelscited by 17
  27. 2026Spherical Steering: Geometry-Aware Activation Rotation for Language Modelscited by 17
  28. 2025Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Modelscited by 16
  29. 2024A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoningcited by 15
  30. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 15
  31. 2025Differentially Private Steering for Large Language Model Alignmentcited by 15
  32. 2026Observing and Controlling Features in Vision-Language-Action Modelscited by 14
  33. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 13
  34. 2025EasySteer: A Unified Framework for High-Performance and Extensible LLM Steeringcited by 13
  35. 2024Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Modelscited by 12
  36. 2025LinEAS: End-to-end Learning of Activation Steering with a Distributional Losscited by 12
  37. 2025Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruningcited by 12
  38. 2024Test-Time Alignment via Hypothesis Reweightingcited by 11

50 most cited of 186 All 186 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning
  2. 2025Efficient Post-Training Refinement of Latent Reasoning in Large Language Models
  3. 2025PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models
  4. 2026Echo-LoRA: Parameter-Efficient Fine-Tuning via Cross-Layer Representation Injection
  5. 2025Balancing the Budget: Understanding Trade-offs Between Supervised and Preference-Based Finetuning
  6. 2023REFINER: Reasoning Feedback on Intermediate Representations
  7. 2025Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate
  8. 2025Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning