mlsanityData to 14 Oct 2026

Measuring Association Between Labels and Free-Text Rationales

arXiv 2010.12762 ↗First posted 24 Oct 2020cs.CLcited by 160

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investigate their less-studied counterpart: free-text natural language rationales. We demonstrate that pipelines, existing models for faithful extractive rationalization on information-extraction style tasks, do not extend as reliably to "reasoning" tasks requiring free-text rationales. We turn to models that jointly predict and rationalize, a class of widely used high-performance models for free-text rationalization whose faithfulness is not yet established. We define label-rationale association as a necessary property for faithfulness: the internal mechanisms of the model producing the label and the rationale must be meaningfully correlated. We propose two measurements to test this property: robustness equivalence and feature importance agreement. We find that state-of-the-art T5-based joint models exhibit both properties for rationalizing commonsense question-answering and natural language inference, indicating their potential for producing faithful free-text rationales.

Key facts. Cited by 160 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 27; 2023: 43; 2024: 31; 2025: 25; 2026: 17.

160 papers.

Sort:

By area:

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  3. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  4. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 196
  5. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  6. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  7. 2023Training Language Models with Language Feedback at Scalecited by 128
  8. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  9. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  10. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 124
  11. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 111
  12. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 110
38 more
  1. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 108
  2. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  3. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  4. 2023Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanationscited by 85
  5. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  6. 2022FLUTE: Figurative Language Understanding through Textual Explanationscited by 79
  7. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  8. 2022PINTO: Faithful Language Reasoning Using Prompt-Generated Rationalescited by 56
  9. 2022Training Language Models with Language Feedbackcited by 49
  10. 2024Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillationcited by 47
  11. 2021Prompting Contrastive Explanations for Commonsense Reasoning Taskscited by 45
  12. 2021SelfExplain: A Self-Explaining Architecture for Neural Text Classifierscited by 44
  13. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  14. 2023Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationalescited by 37
  15. 2022Can Rationalization Improve Robustness?cited by 36
  16. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 36
  17. 2023Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Modelscited by 35
  18. 2023COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statementscited by 34
  19. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  20. 2023HARE: Explainable Hate Speech Detection with Step-by-Step Reasoningcited by 31
  21. 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?cited by 29
  22. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  23. 2022E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoningcited by 28
  24. 2023Explanation-based Finetuning Makes Models More Robust to Spurious Cuescited by 25
  25. 2025AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computationcited by 25
  26. 2021Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanationscited by 24
  27. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  28. 2024Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?cited by 24
  29. 2021Rissanen Data Analysis: Examining Dataset Characteristics via Description Lengthcited by 22
  30. 2022REV: Information-Theoretic Evaluation of Free-Text Rationalescited by 22
  31. 2023Rationalization for Explainable NLP: A Surveycited by 22
  32. 2023Sociocultural Norm Similarities and Differences via Situational Alignment and Explainable Textual Entailmentcited by 22
  33. 2024Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMscited by 22
  34. 2022An Empirical Study on Explanations in Out-of-Domain Settingscited by 21
  35. 2022ExPUNations: Augmenting Puns with Keywords and Explanationscited by 21
  36. 2021Connecting Attributions and QA Model Behavior on Realistic Counterfactualscited by 20
  37. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  38. 2023When Do Program-of-Thoughts Work for Reasoning?cited by 19

50 most cited of 160 All 160 (JSON)