mlsanityData to 7 Oct 2026

Mitigating Copy Bias in In-Context Learning through Neuron Pruning

arXiv 2410.01288 ↗First posted 2 Oct 2024cs.CL cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive few-shot in-context learning (ICL) abilities. Still, we show that they are sometimes prone to a `copying bias', where they copy answers from provided examples instead of learning the underlying patterns. In this work, we propose a novel and simple method to mitigate such copying bias. First, we create a synthetic task and use the Integrated Gradients method to identify neurons that prioritize copying over generalization. We demonstrate that pruning these neurons consistently improves performance across a diverse set of ICL tasks. We also show that our method is applicable across various LLM architectures, including Transformers and State-Space Models, without requiring modifications. In our analysis, we adopt a task-recognition perspective on ICL and examine task vectors (Hendel et al., 2023) induced by the model. We find that pruning enhances the quality of these vectors, suggesting that the pruned neurons previously hindered effective task recognition.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

43 of 48 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  11. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  12. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
31 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  4. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  5. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  6. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  8. 2022In-context Learning and Induction Headscited by 930
  9. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  10. 2022Transformers learn in-context by gradient descentcited by 714
  11. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  12. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  13. 2023In-Context Learning Creates Task Vectorscited by 302
  14. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  15. 2022Active Example Selection for In-Context Learningcited by 195
  16. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  17. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 124
  18. 2024Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Taskscited by 104
  19. 2024Universal Neurons in GPT2 Language Modelscited by 100
  20. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  21. 2022An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labelscited by 88
  22. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  23. 2022Structured Prompting: Scaling In-Context Learning to 1,000 Examplescited by 81
  24. 2023Symbol tuning improves in-context learning in language modelscited by 72
  25. 2022Prototypical Calibration for Few-shot Learning of Language Modelscited by 58
  26. 2024Confidence Regulation Neurons in Language Modelscited by 57
  27. 2024Is Mamba Capable of In-Context Learning?cited by 45
  28. 2024IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neuronscited by 36
  29. 2024MEND: Meta dEmonstratioN Distillation for Efficient and Effective In-Context Learningcited by 15
  30. 2023Fine-tune Language Models to Approximate Unbiased In-context Learningcited by 12
  31. 2023Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 5.

11 papers.

  1. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoningcited by 16
  2. 2024Shortcut Learning in In-Context Learning: A Surveycited by 8
  3. 2026Counterfactual Trace Auditing of LLM Agent Skillscited by 8
  4. 2025In-Context Learning (and Unlearning) of Length Biasescited by 3
  5. 2025Detecting and Pruning Prominent but Detrimental Neurons in Large Language Modelscited by 2
  6. 2025Don't Think of the White Bear: Ironic Negation in Transformer Models Under Cognitive Loadcited by 1
  7. 2026When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Modelscited by 1
  8. 2026LLM Parameters for Math Across Languages: Shared or Separate?cited by 1
  9. 2025Beyond Plain Demos: A Demo-centric Anchoring Paradigm for In-Context Learning in Alzheimer's Disease Detectioncited by 0
  10. 2026When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscriptscited by 0
  11. 2026Towards Better Exploration in Sequential Test-Time Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Exploring the Limits of Pruning: Task-Specific Neurons, Model Collapse, and Recovery in Task-Specific Large Language Models
  2. 2024Language Models "Grok" to Copy
  3. 2023Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
  4. 2025Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models
  5. 2022Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale
  6. 2026Batch-wise Adaptive Pruning: Periodic Neuron Activation-Aware Weight Pruning for Language Reasoning Model
  7. 2025From Compression to Expression: A Layerwise Analysis of In-Context Learning
  8. 2024Many-Shot In-Context Learning