# Teach me how to Label: Labeling Functions from Natural Language with Text-to-text Transformers

arXiv: https://arxiv.org/abs/2101.07138 · HTML: https://mlsanity.com/paper/2101.07138/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow](https://mlsanity.com/paper/1805.08949/) (2018)
- [Training Classifiers with Natural Language Explanations](https://mlsanity.com/paper/1805.03818/) (2018)
- [Incorporating External Knowledge through Pre-training for Natural Language to Code Generation](https://mlsanity.com/paper/2004.09015/) (2020)
- [ALICE: Active Learning with Contrastive Natural Language Explanations](https://mlsanity.com/paper/2009.10259/) (2020)
- [StaQC: A Systematically Mined Question-Code Dataset from Stack Overflow](https://mlsanity.com/paper/1803.09371/) (2018)
- [TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition](https://mlsanity.com/paper/2004.07493/) (2020)
- [ExpBERT: Representation Engineering with Natural Language Explanations](https://mlsanity.com/paper/2005.01932/) (2020)
- [NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction](https://mlsanity.com/paper/1909.02177/) (2019)
- [Teaching Machine Comprehension with Compositional Explanations](https://mlsanity.com/paper/2005.00806/) (2020)
- [Learning from Explanations with Neural Execution Tree](https://mlsanity.com/paper/1911.01352/) (2019)

## Abstract (arXiv)

Annotated data has become the most important bottleneck in training accurate machine learning models, especially for areas that require domain expertise. A recent approach to deal with the above issue proposes using natural language explanations instead of labeling individual data points, thereby increasing human annotators' efficiency as well as decreasing costs substantially. This paper focuses on the task of turning these natural language descriptions into Python labeling functions by following a novel approach to semantic parsing with pre-trained text-to-text Transformers. In a series of experiments our approach achieves a new state of the art on the semantic parsing benchmark CoNaLa, surpassing the previous best approach by 3.7 BLEU points. Furthermore, on a manually constructed dataset of natural language descriptions-labeling functions pairs we achieve a BLEU of 0.39. Our approach can be regarded as a stepping stone towards models that are taught how to label in natural language, instead of being provided specific labeled samples. Our code, constructed dataset and models are available at https://github.com/ypapanik/t5-for-code-generation.