mlsanityData to 7 Oct 2026

A Multilingual Perspective Towards the Evaluation of Attribution Methods in Natural Language Inference

arXiv 2204.05428 ↗First posted 11 Apr 2022cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Most evaluations of attribution methods focus on the English language. In this work, we present a multilingual approach for evaluating attribution methods for the Natural Language Inference (NLI) task in terms of faithfulness and plausibility. First, we introduce a novel cross-lingual strategy to measure faithfulness based on word alignments, which eliminates the drawbacks of erasure-based evaluations.We then perform a comprehensive evaluation of attribution methods, considering different output mechanisms and aggregation methods. Finally, we augment the XNLI dataset with highlight-based explanations, providing a multilingual NLI dataset with highlights, to support future exNLP studies. Our results show that attribution methods performing best for plausibility and faithfulness are different.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 35 of its 55 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

35 of 55 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  10. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
23 more
  1. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  2. 2019Attention is not Explanationcited by 916
  3. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  4. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  5. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  6. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  7. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  8. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  9. 2019Is Attention Interpretable?cited by 387
  10. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  11. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  12. 2015Visualizing and Understanding Recurrent Networkscited by 158
  13. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 158
  14. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  15. 2016"What is Relevant in a Text Document?": An Interpretable Machine Learning Approachcited by 53
  16. 2020Interpretation of NLP models through input marginalizationcited by 43
  17. 2018Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Valuescited by 40
  18. 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
  19. 2021Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retrainingcited by 31
  20. 2021Relative Importance in Sentence Processingcited by 22
  21. 2021Translation Error Detection as Rationale Extractioncited by 22
  22. 2020To what extent do human explanations of model behavior align with actual model behavior?cited by 18
  23. 2021Fine-grained Interpretation and Causation Analysis in Deep NLP Modelscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 5.

5 papers.

  1. 2023Explaining How Transformers Use Context to Build Predictionscited by 42
  2. 2023The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metricscited by 15
  3. 2023Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictionscited by 6
  4. 2023Faithfulness Measurable Masked Language Modelscited by 5
  5. 2023X-PARADE: Cross-Lingual Textual Entailment and Information Divergence across Paragraphscited by 3