mlsanityData to 7 Oct 2026

SAKE: Steering Activations for Knowledge Editing

arXiv 2503.01751 ↗First posted 3 Mar 2025cs.AI cs.CL cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

As Large Langue Models have been shown to memorize real-world facts, the need to update this knowledge in a controlled and efficient manner arises. Designed with these constraints in mind, Knowledge Editing (KE) approaches propose to alter specific facts in pretrained models. However, they have been shown to suffer from several limitations, including their lack of contextual robustness and their failure to generalize to logical implications related to the fact. To overcome these issues, we propose SAKE, a steering activation method that models a fact to be edited as a distribution rather than a single prompt. Leveraging Optimal Transport, SAKE alters the LLM behavior over a whole fact-related distribution, defined as paraphrases and logical implications. Several numerical experiments demonstrate the effectiveness of this method: SAKE is thus able to perform more robust edits than its existing counterparts.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

30 of 47 references.

References · 30

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2022Mass-Editing Memory in a Transformercited by 976
  8. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2021Fast Model Editing at Scalecited by 491
  11. 2022Memory-Based Model Editing at Scalecited by 443
  12. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
18 more
  1. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  2. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  3. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  4. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  5. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  6. 2020Modifying Memories in Transformer Modelscited by 222
  7. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  8. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  9. 2023Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluationcited by 134
  10. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  11. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  12. 2024Controlling Language and Diffusion Models by Transporting Activationscited by 59
  13. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
  14. 2024WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editingcited by 40
  15. 2024Uncovering Overfitting in Large Language Model Editingcited by 36
  16. 2023Evaluating the Reversal Curse in Model Editingcited by 34
  17. 2024Model Editing by Standard Fine-Tuningcited by 22
  18. 2021Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddingscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 9.

12 papers.

  1. 2025EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Modelscited by 7
  2. 2026Steering Language Models Before They Speak: Logit-Level Interventionscited by 3
  3. 2026Where Steering Signals Come From: Activation Source Selection in Activation Steeringcited by 2
  4. 2025MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMscited by 1
  5. 2025EtCon: Edit-then-Consolidate for Reliable Knowledge Editingcited by 1
  6. 2026Targeted Recovery of Weight-Space Mechanisms From Neural Networkscited by 1
  7. 2026The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editingcited by 0
  8. 2026Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMscited by 0
  9. 2026Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editingcited by 0
  10. 2026xMIx: High-Performance Serving-Time Platform for Mechanistic Interpretability Appscited by 0
  11. 2026Toward Latent Language Model Skills Steering and Optimization: An Empirical Studycited by 0
  12. 2026IMPLICIT-Bench: Measuring Implicit Bias in Text-to-Image Models under Neutral Promptscited by 0