mlsanityData to 7 Oct 2026

Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders

arXiv 2406.19309 ↗First posted 27 Jun 2024cs.IRcited by 1

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

With the recent addition of Retrieval-Augmented Generation (RAG), the scope and importance of Information Retrieval (IR) has expanded. As a result, the importance of a deeper understanding of IR models also increases. However, interpretability in IR remains under-explored, especially when it comes to the models' inner mechanisms. In this paper, we explore the possibility of adapting Integrated Gradient-based methods in an IR context to identify the role of individual neurons within the model. In particular, we provide new insights into the role of what we call "relevance" neurons, as well as how they deal with unseen data. Finally, we carry out an in-depth pruning study to validate our findings.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 34 of its 58 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

34 of 58 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  12. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
22 more
  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  2. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  3. 2022In-context Learning and Induction Headscited by 930
  4. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  5. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  6. 2019Passage Re-ranking with BERTcited by 629
  7. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  8. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  9. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  10. 2020Overview of the TREC 2019 deep learning trackcited by 282
  11. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  12. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  13. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  14. 2018How Important Is a Neuron?cited by 73
  15. 2022A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributionscited by 64
  16. 2019How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representationscited by 63
  17. 2023Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neuronscited by 56
  18. 2020Declarative Experimentation in Information Retrieval using PyTerriercited by 29
  19. 2021Simplified Data Wrangling with ir_datasetscited by 27
  20. 2020TREC-COVID: Constructing a Pandemic Information Retrieval Test Collectioncited by 23
  21. 2018Computationally Efficient Measures of Internal Neuron Importancecited by 19
  22. 2023An Attribution Method for Siamese Encoderscited by 8

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Explaining Caption-Image Interactions in CLIP Models with Second-Order Attributionscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026Neurosymbolic Retrievers for Retrieval-augmented Generation
  2. 2026Towards a Relevance Posterior in Neural Information Access ↗
  3. 2026SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation
  4. 2026Self-Aware Vector Embeddings for Retrieval-Augmented Generation: A Neuroscience-Inspired Framework for Temporal, Confidence-Weighted, and Relational Knowledge
  5. 2023Large Language Models for Information Retrieval: A Survey
  6. 2025NDAI-NeuroMAP: A Neuroscience-Specific Embedding Model for Domain-Specific Retrieval
  7. 2026Retrieval from Within: An Intrinsic Capability of Attention-Based Models
  8. 2024ir_explain: a Python Library of Explainable IR Methods ↗