mlsanityData to 14 Oct 2026

Influence Functions for Sequence Tagging Models

arXiv 2210.14177 ↗First posted 25 Oct 2022cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Many language tasks (e.g., Named Entity Recognition, Part-of-Speech tagging, and Semantic Role Labeling) are naturally framed as sequence tagging problems. However, there has been comparatively little work on interpretability methods for sequence tagging models. In this paper, we extend influence functions - which aim to trace predictions back to the training points that informed them - to sequence tagging tasks. We define the influence of a training instance segment as the effect that perturbing the labels within this segment has on a test segment level prediction. We provide an efficient approximation to compute this, and show that it tracks with the true segment influence, measured empirically. We show the practical utility of segment influence by using the method to identify systematic annotation errors in two named entity recognition corpora. Code to reproduce our results is available at https://github.com/successar/Segment_Influence_Functions.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 29 of its 60 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

29 of 60 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  5. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  6. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  7. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  8. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  9. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  10. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  11. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  12. 2021Scaling Up Influence Functionscited by 132
17 more
  1. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  2. 2018Understanding the Origins of Bias in Word Embeddingscited by 91
  3. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  4. 2018A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literaturecited by 79
  5. 2017A causal framework for explaining the predictions of black-box sequence-to-sequence modelscited by 58
  6. 2018Interpreting Black Box Predictions using Fisher Kernelscited by 48
  7. 2021HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networkscited by 38
  8. 2021Combining Feature and Instance Attribution to Detect Artifactscited by 38
  9. 2021An Empirical Comparison of Instance Attribution Methods for NLPcited by 35
  10. 2021Influence Tuning: Demoting Spurious Correlations via Instance Attribution and Instance-Driven Updatescited by 27
  11. 2021Rationales for Sequential Predictionscited by 26
  12. 2020Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improvecited by 16
  13. 2021The Trade-offs of Domain Adaptation for Neural Language Modelscited by 14
  14. 2021On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluationcited by 12
  15. 2020Interpreting Robust Optimization via Adversarial Influence Functionscited by 8
  16. 2021Gradient-guided Loss Masking for Neural Machine Translationcited by 8
  17. 2019Understanding Goal-Oriented Active Learning via Influence Functionscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 1; 2025: 1; 2026: 2.

9 papers.

  1. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  2. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  3. 2023How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuningcited by 14
  4. 2023CleanCoNLL: A Nearly Noise-Free Named Entity Recognition Datasetcited by 11
  5. 2023Deeper Understanding of Black-box Predictions via Generalized Influence Functionscited by 7
  6. 2026Step-resolved data attribution for looped transformerscited by 5
  7. 2023Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retrainingcited by 4
  8. 2025Faithful and Fast Influence Function via Advanced Samplingcited by 0
  9. 2026MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignmentcited by 0