mlsanityData to 14 Oct 2026

Interpreting Multilingual and Document-Length Sensitive Relevance Computations in Neural Retrieval Models through Axiomatic Causal Interventions

arXiv 2505.02154 ↗First posted 4 May 2025cs.IR cs.AIcited by 0

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

This reproducibility study analyzes and extends the paper "Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models," which investigates how neural retrieval models encode task-relevant properties such as term frequency. We reproduce key experiments from the original paper, confirming that information on query terms is captured in the model encoding. We extend this work by applying activation patching to Spanish and Chinese datasets and by exploring whether document-length information is encoded in the model as well. Our results confirm that the designed activation patching method can isolate the behavior to specific components and tokens in neural retrieval models. Moreover, our findings indicate that the location of term frequency generalizes across languages and that in later layers, the information for sequence-level tasks is represented in the CLS token. The results highlight the need for further research into interpretability in information retrieval and reproducibility in machine learning research. Our code is available at https://github.com/OliverSavolainen/axiomatic-ir-reproduce.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 10 of its 19 references that are in the map. First posted on arXiv 4 May 2025.

Builds on

10 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methods27 Sep

Other references · 9

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  5. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  6. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  7. 2024Systematic Evaluation of Neural Retrieval Models on the Touch'e 2020 Argument Retrieval Subset of BEIRcited by 9
  8. 2024Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Modelscited by 8
  9. 2025MechIR: A Mechanistic Interpretability Framework for Information Retrievalcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective
  2. 2026Why Large Language Models can Secretly Outperform Embedding Similarity in Information Retrieval ↗
  3. 2025CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation
  4. 2026Towards a Relevance Posterior in Neural Information Access ↗
  5. 2023Large Language Models for Information Retrieval: A Survey
  6. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Context
  7. 2025Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Models
  8. 2024Probing Causality Manipulation of Large Language Models