# Correcting Large Language Model Behavior via Influence Function

arXiv: https://arxiv.org/abs/2412.16451 · HTML: https://mlsanity.com/paper/2412.16451/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 12 of its 17 references that are in the map.
- First posted on arXiv 21 Dec 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://mlsanity.com/paper/2210.01241/) (2022)
- [Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis](https://mlsanity.com/paper/1806.03884/) (2018)
- [Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures](https://mlsanity.com/paper/2311.00636/) (2023)

## Built on by (most cited first, 9 of 9)

- [Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages](https://mlsanity.com/paper/2603.12658/) (2026)
- [First is Not Really Better Than Last: Evaluating Layer Choice and Aggregation Strategies in Language Model Data Influence Estimation](https://mlsanity.com/paper/2511.04715/) (2025)
- [Influence Functions for Edge Edits in Non-Convex Graph Neural Networks](https://mlsanity.com/paper/2506.04694/) (2025)
- [Newfluence: Boosting Model interpretability and Understanding in High Dimensions](https://mlsanity.com/paper/2507.11895/) (2025)
- [Soft Weighted Machine Unlearning](https://mlsanity.com/paper/2505.18783/) (2025)
- [Data Security in Large Language Models: Risks, Defense, and Directions](https://mlsanity.com/paper/2508.02312/) (2025)
- [The Realignment Problem: When Right becomes Wrong in LLMs](https://mlsanity.com/paper/2511.02623/) (2025)
- [Scalable Attribution and Control of Model Behavior During Training](https://mlsanity.com/paper/2609.33667/) (2026)
- [Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection](https://mlsanity.com/paper/2609.37842/) (2026)

## Abstract (arXiv)

Recent advancements in AI alignment techniques have significantly improved the alignment of large language models (LLMs) with static human preferences. However, the dynamic nature of human preferences can render some prior training data outdated or even erroneous, ultimately causing LLMs to deviate from contemporary human preferences and societal norms. Existing methodologies, whether they involve the curation of new data for continual alignment or the manual correction of outdated data for re-alignment, demand costly human resources. To address this challenge, we propose a novel approach, Large Language Model Behavior Correction with Influence Function Recall and Post-Training (LANCET), which requires no human involvement. LANCET consists of two phases: (1) using influence functions to identify the training data that significantly impact undesirable model outputs, and (2) applying an Influence function-driven Bregman Optimization (IBO) technique to adjust the model's behavior based on these influence distributions. Our experiments demonstrate that LANCET effectively and efficiently correct inappropriate behaviors of LLMs. Furthermore, LANCET can outperform methods that rely on collecting human preferences, and it enhances the interpretability of learning human preferences within LLMs.