mlsanityData to 7 Oct 2026

Towards a Framework for Evaluating Explanations in Automated Fact Verification

arXiv 2403.20322 ↗First posted 29 Mar 2024cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

As deep neural models in NLP become more complex, and as a consequence opaque, the necessity to interpret them becomes greater. A burgeoning interest has emerged in rationalizing explanations to provide short and coherent justifications for predictions. In this position paper, we advocate for a formal framework for key concepts and properties about rationalizing explanations to support their evaluation systematically. We also outline one such formal framework, tailored to rationalizing explanations of increasingly complex structures, from free-form explanations to deductive explanations, to argumentative explanations (with the richest structure). Focusing on the automated fact verification task, we provide illustrations of the use and usefulness of our formalization for evaluating explanations, tailored to their varying structures.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 46 of its 68 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

46 of 68 references.

References · 46

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  4. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  6. 2019Attention is not Explanationcited by 916
  7. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  8. 2019Attention is not not Explanationcited by 629
  9. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  10. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  11. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  12. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
34 more
  1. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  2. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  3. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  4. 2016Rationalizing Neural Predictionscited by 281
  5. 2022Can language models learn from explanations in context?cited by 248
  6. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  7. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  8. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  9. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 158
  10. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  11. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  12. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  13. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  14. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 105
  15. 2020NILE : Natural Language Inference with Faithful Natural Language Explanationscited by 96
  16. 2020Explainable Automated Fact-Checking: A Surveycited by 83
  17. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  18. 2017A Stylometric Inquiry into Hyperpartisan and Fake Newscited by 81
  19. 2017SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumourscited by 77
  20. 2018RumourEval 2019: Determining Rumour Veracity and Support for Rumourscited by 74
  21. 2018Contrastive Explanation: A Structural-Model Approachcited by 64
  22. 2021ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoningcited by 58
  23. 2021ProoFVer: Natural Logic Theorem Proving for Fact Verificationcited by 52
  24. 2020Universal Adversarial Attacks with Natural Triggers for Text Classificationcited by 50
  25. 2019R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reasoncited by 36
  26. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  27. 2021Explainable AI: current status and future directionscited by 32
  28. 2021Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classificationcited by 29
  29. 2020Generating Label Cohesive and Well-Formed Adversarial Claimscited by 27
  30. 2022Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learningcited by 15
  31. 2021Flexible Instance-Specific Rationalization of NLP Modelscited by 10
  32. 2022PHEMEPlus: Enriching Social Media Rumour Verification with External Evidencecited by 6
  33. 2022RerrFact: Reduced Evidence Retrieval Representations for Scientific Claim Verificationcited by 5
  34. 2021Interpretability in Safety-Critical FinancialTrading Systemscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2024Argumentative Large Language Models for Explainable and Contestable Claim Verificationcited by 36
  2. 2024CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structurescited by 11
  3. 2025Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMscited by 5
  4. 2026Responsible Evaluation of AI for Mental Healthcited by 2
  5. 2025Pub-Guard-LLM: Detecting Retracted Biomedical Articles with Reliable Explanationscited by 1
  6. 2025FinGrAct: A Framework for FINe-GRrained Evaluation of ACTionability in Explainable Automatic Fact-Checkingcited by 1
  7. 2026SoK: Formal Methods for Fact-Checking and Information Integritycited by 0