mlsanityData to 14 Oct 2026

Few-Shot Self-Rationalization with Natural Language Prompts

arXiv 2111.08284 ↗First posted 16 Nov 2021cs.CLcited by 83

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These models are, however, currently trained with a large amount of human-written free-text explanations for each task which hinders their broader usage. We propose to study a more realistic setting of self-rationalization using few training examples. We present FEB -- a standardized collection of four existing English-language datasets and associated metrics. We identify the right prompting approach by extensively exploring natural language prompts on FEB. Then, by using this prompt and scaling the model size, we demonstrate that making progress on few-shot self-rationalization is possible. We show there is still ample room for improvement in this task: the average plausibility of generated explanations assessed by human annotators is at most 51% (with GPT-3), while plausibility of human explanations is 76%. We hope that FEB and our proposed approach will spur the community to take on the few-shot self-rationalization challenge.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 16 Nov 2021.

Builds on

49 of 74 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  12. 2019How Can We Know What Language Models Know?cited by 959
37 more
  1. 2018Neural Network Acceptability Judgmentscited by 958
  2. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  3. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  4. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  5. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  6. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  7. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  8. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  9. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 377
  10. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  11. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  12. 2016Rationalizing Neural Predictionscited by 281
  13. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  14. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 211
  15. 2018ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehensioncited by 200
  16. 2020Measuring Association Between Labels and Free-Text Rationalescited by 160
  17. 2016Generating Visual Explanationscited by 158
  18. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  19. 2019Interpretable Neural Predictions with Differentiable Binary Variablescited by 126
  20. 2019Rethinking Cooperative Rationalization: Introspective Extraction and Complement Controlcited by 96
  21. 2020Compressing Large-Scale Transformer-Based Models: A Case Study on BERTcited by 96
  22. 2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
  23. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  24. 2020An Information Bottleneck Approach for Controlling Conciseness in Rationale Extractioncited by 69
  25. 2021FLEX: Unifying Evaluation for Few-Shot NLPcited by 67
  26. 2018Faithful Multimodal Explanation for Visual Question Answeringcited by 64
  27. 2018Training Classifiers with Natural Language Explanationscited by 58
  28. 2018Towards Explainable NLP: A Generative Explanation Framework for Text Classificationcited by 55
  29. 2020Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphscited by 46
  30. 2019Does It Make Sense? And Why? A Pilot Study for Sense Making and Explanationcited by 43
  31. 2021Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answeringcited by 41
  32. 2017Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanationscited by 37
  33. 2020Evaluating and Characterizing Human Rationalescited by 36
  34. 2020Explaining Question Answering Models through Text Generationcited by 33
  35. 2019Evaluating Explanation Without Ground Truth in Interpretable Machine Learningcited by 29
  36. 2020Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervisioncited by 27
  37. 2019Question Answering is a Format; When is it Useful?cited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 19; 2023: 31; 2024: 17; 2025: 13; 2026: 1.

83 papers.

Sort:

By area:

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  3. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  4. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  5. 2022Can language models learn from explanations in context?cited by 249
  6. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
  7. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  8. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 154
  9. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
  10. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  11. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  12. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
38 more
  1. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 119
  2. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  3. 2023Text Classification via Large Language Modelscited by 106
  4. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  5. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  6. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 84
  7. 2022Parsel: Algorithmic Reasoning with Language Models by Composing Decompositionscited by 64
  8. 2022PINTO: Faithful Language Reasoning Using Prompt-Generated Rationalescited by 56
  9. 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoningcited by 51
  10. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  11. 2022CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negationcited by 39
  12. 2025Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistenciescited by 39
  13. 2023Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationalescited by 37
  14. 2024The broader spectrum of in-context learningcited by 35
  15. 2023Interactive Natural Language Processingcited by 31
  16. 2023HARE: Explainable Hate Speech Detection with Step-by-Step Reasoningcited by 31
  17. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  18. 2024Source-Aware Training Enables Knowledge Attribution in Language Modelscited by 30
  19. 2023Second Thoughts are Best: Learning to Re-Align With Human Values from Text Editscited by 27
  20. 2023Explanation-based Finetuning Makes Models More Robust to Spurious Cuescited by 25
  21. 2024Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervisioncited by 25
  22. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 25
  23. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  24. 2025VeriPlan: Integrating Formal Verification and LLMs into End-User Planningcited by 20
  25. 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
  26. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  27. 2021Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setupcited by 16
  28. 2023Distilling ChatGPT for Explainable Automated Student Answer Assessmentcited by 14
  29. 2023Self-Contradictory Reasoning Evaluation and Detectioncited by 14
  30. 2022Investigating the Benefits of Free-Form Rationalescited by 13
  31. 2023Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanationscited by 13
  32. 2023A Study of Situational Reasoning for Traffic Understandingcited by 13
  33. 2025The Dead Salmons of AI Interpretabilitycited by 12
  34. 2023InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanationscited by 11
  35. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11
  36. 2023FLamE: Few-shot Learning from Natural Language Explanationscited by 10
  37. 2022Calibrating Trust of Multi-Hop Question Answering Systems with Decompositional Probescited by 9
  38. 2023Characterizing Large Language Models as Rationalizers of Knowledge-intensive Taskscited by 9

50 most cited of 83 All 83 (JSON)