Training Classifiers with Natural Language Explanations
Not on the map. Why?
Abstract (arXiv)
Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100× faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.
Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 May 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2020: 1; 2021: 19; 2022: 18; 2023: 10; 2024: 6; 2025: 2; 2026: 2.
58 papers.
Sort:
By area:
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
- 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
- 2021Explaining Answers with Entailment Treescited by 152
- 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
- 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
- 2022Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoningcited by 79
- 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
- 2023Learning to Generate Better Than Your LLMcited by 49
- 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoningcited by 47
- 2020Learning Rewards from Linguistic Feedbackcited by 41
38 more
- 2021Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similaritycited by 35
- 2022Fixing Model Bugs with Natural Language Patchescited by 33
- 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
- 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
- 2021On the Diversity and Limits of Human Explanationscited by 30
- 2021Refining Language Models with Compositional Explanationscited by 27
- 2021What to Learn, and How: Toward Effective Learning from Rationalescited by 25
- 2021Teaching Autoregressive Language Models Complex Tasks By Demonstrationcited by 15
- 2022A survey on improving NLP models with human explanationscited by 15
- 2022PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learningcited by 14
- 2022Factual and Informative Review Generation for Explainable Recommendationcited by 14
- 2021Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Healthcited by 13
- 2022Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methodscited by 13
- 2023FLamE: Few-shot Learning from Natural Language Explanationscited by 10
- 2021HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversationscited by 9
- 2022CLUES: A Benchmark for Learning Classifiers using Natural Language Explanationscited by 9
- 2022Does Self-Rationalization Improve Robustness to Spurious Correlations?cited by 9
- 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
- 2022Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detectioncited by 7
- 2024INTERACT: Enabling Interactive, Question-Driven Learning in Large Language Modelscited by 6
- 2022It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifierscited by 5
- 2023BYOC: Personalized Few-Shot Classification with Co-Authored Class Descriptionscited by 5
- 2021TagRuler: Interactive Tool for Span-Level Data Programming by Demonstrationcited by 4
- 2022Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Predictioncited by 4
- 2022Zero-Shot Classification by Logical Reasoning on Natural Language Explanationscited by 4
- 2023Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarioscited by 4
- 2023Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproductscited by 4
- 2021D-REX: Dialogue Relation Extraction with Explanationscited by 3
- 2022SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalizationcited by 2
- 2022LaSQuE: Improved Zero-Shot Classification from Explanations Through Quantifier Modeling and Curriculum Learningcited by 2
- 2023Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifierscited by 2
- 2023What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perceptioncited by 2
- 2021AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extractioncited by 1
- 2021Open Rule Inductioncited by 1
- 2022Label Augmentation with Reinforced Labeling for Weak Supervisioncited by 1
- 2024ActiveDP: Bridging Active Learning and Data Programmingcited by 1
- 2024WeShap: Weak Supervision Source Evaluation with Shapley Valuescited by 1
- 2024A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)cited by 1
50 most cited of 58 All 58 (JSON)
- 2026Training-Time Explainability for Multilingual Hate Speech Detection: Aligning Model Reasoning with Human Rationales22 Jun 2026
- 2026EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision1 Jun 2026
- 2025Socratic Students: Teaching Language Models to Learn by Asking Questions15 Dec 2025
- 2025Refining Labeling Functions with Limited Labeled Data29 May 2025
- 2024INTERACT: Enabling Interactive, Question-Driven Learning in Large Language Models16 Dec 2024
- 2024An Approach for Auto Generation of Labeling Functions for Software Engineering Chatbots9 Oct 2024
- 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLP3 Jul 2024
- 2024A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)27 Jun 2024
- 2024WeShap: Weak Supervision Source Evaluation with Shapley Values16 Jun 2024
- 2024ActiveDP: Bridging Active Learning and Data Programming8 Feb 2024
- 2023What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception16 Nov 2023
- 2023Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers13 Nov 2023
38 more
- 2023BYOC: Personalized Few-Shot Classification with Co-Authored Class Descriptions9 Oct 2023
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines5 Oct 2023
- 2023Learning to Generate Better Than Your LLM20 Jun 2023
- 2023FLamE: Few-shot Learning from Natural Language Explanations13 Jun 2023
- 2023Exploring Automatically Perturbed Natural Language Explanations in Relation Extraction24 May 2023
- 2023Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts30 Mar 2023
- 2023Large Language Model Instruction Following: A Survey of Progresses and Challenges18 Mar 2023
- 2023Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarios21 Feb 2023
- 2022LaSQuE: Improved Zero-Shot Classification from Explanations Through Quantifier Modeling and Curriculum Learning18 Dec 2022
- 2022Fixing Model Bugs with Natural Language Patches7 Nov 2022
- 2022Zero-Shot Classification by Logical Reasoning on Natural Language Explanations7 Nov 2022
- 2022Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detection1 Nov 2022
- 2022Does Self-Rationalization Improve Robustness to Spurious Correlations?24 Oct 2022
- 2022Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods13 Oct 2022
- 2022Factual and Informative Review Generation for Explainable Recommendation12 Sep 2022
- 2022Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction28 Jun 2022
- 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoning12 May 2022
- 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning6 May 2022
- 2022It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifiers25 Apr 2022
- 2022A survey on improving NLP models with human explanations19 Apr 2022
- 2022CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations14 Apr 2022
- 2022Label Augmentation with Reinforced Labeling for Weak Supervision13 Apr 2022
- 2022PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning18 Mar 2022
- 2022SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalization26 Feb 2022
- 2022Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoning17 Feb 2022
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan 2022
- 2021Technical Language Supervision for Intelligent Fault Diagnosis in Process Industry11 Dec 2021
- 2021What to Learn, and How: Toward Effective Learning from Rationales30 Nov 2021
- 2021Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity16 Nov 2021
- 2021Few-Shot Self-Rationalization with Natural Language Prompts16 Nov 2021
- 2021Open Rule Induction26 Oct 2021
- 2021Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health28 Sep 2021
- 2021D-REX: Dialogue Relation Extraction with Explanations10 Sep 2021
- 2021AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction10 Sep 2021
- 2021Teaching Autoregressive Language Models Complex Tasks By Demonstration5 Sep 2021
- 2021TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration24 Jun 2021
- 2021On the Diversity and Limits of Human Explanations22 Jun 2021
- 2021HERALD: An Annotation Efficient Method to Detect User Disengagement in Social Conversations1 Jun 2021
50 newest of 58 All 58 (JSON)
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
- 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoningcited by 47
- 2022Zero-Shot Classification by Logical Reasoning on Natural Language Explanationscited by 4
- 2022Zero-Shot Classification by Logical Reasoning on Natural Language Explanations7 Nov 2022
- 2022e-CARE: a New Dataset for Exploring Explainable Causal Reasoning12 May 2022
- 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning6 May 2022
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan 2022
- 2020Learning Rewards from Linguistic Feedbackcited by 41
- 2020Learning Rewards from Linguistic Feedback30 Sep 2020
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021Explaining Answers with Entailment Treescited by 152
- 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
- 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
- 2022Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoningcited by 79
- 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
- 2023Learning to Generate Better Than Your LLMcited by 49
- 2021Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similaritycited by 35
- 2022Fixing Model Bugs with Natural Language Patchescited by 33
- 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
- 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
- 2021On the Diversity and Limits of Human Explanationscited by 30
- 2021Refining Language Models with Compositional Explanationscited by 27
12 most cited of 51 All 51 (JSON)
- 2026Training-Time Explainability for Multilingual Hate Speech Detection: Aligning Model Reasoning with Human Rationales22 Jun 2026
- 2026EvoPool: Evolutionary Programmatic Annotation for Label-Efficient Specialized Supervision1 Jun 2026
- 2025Socratic Students: Teaching Language Models to Learn by Asking Questions15 Dec 2025
- 2025Refining Labeling Functions with Limited Labeled Data29 May 2025
- 2024INTERACT: Enabling Interactive, Question-Driven Learning in Large Language Models16 Dec 2024
- 2024An Approach for Auto Generation of Labeling Functions for Software Engineering Chatbots9 Oct 2024
- 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLP3 Jul 2024
- 2024A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)27 Jun 2024
- 2024WeShap: Weak Supervision Source Evaluation with Shapley Values16 Jun 2024
- 2024ActiveDP: Bridging Active Learning and Data Programming8 Feb 2024
- 2023What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception16 Nov 2023
- 2023Leveraging Multiple Teachers for Test-Time Adaptation of Language-Guided Classifiers13 Nov 2023
12 newest of 51 All 51 (JSON)