# Can Large Language Models Design Accurate Label Functions?

arXiv: https://arxiv.org/abs/2311.00739 · HTML: https://mlsanity.com/paper/2311.00739/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Can Foundation Models Wrangle Your Data?](https://mlsanity.com/paper/2205.09911/) (2022)
- [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods](https://mlsanity.com/paper/2002.11955/) (2020)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale](https://mlsanity.com/paper/1812.00417/) (2018)
- [Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling](https://mlsanity.com/paper/2012.06046/) (2020)
- [Denoising Multi-Source Weak Supervision for Neural Text Classification](https://mlsanity.com/paper/2010.04582/) (2020)
- [Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches](https://mlsanity.com/paper/2211.16285/) (2022)
- [NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction](https://mlsanity.com/paper/1909.02177/) (2019)

## Built on by (most cited first, 6 of 6)

- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [Cost-Effective In-Context Learning for Entity Resolution: A Design Space Exploration](https://mlsanity.com/paper/2312.03987/) (2023)
- [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://mlsanity.com/paper/2407.11004/) (2024)
- [Automated Query-Product Relevance Labeling using Large Language Models for E-commerce Search](https://mlsanity.com/paper/2502.15990/) (2025)
- [LLM-assisted Labeling Function Generation for Semantic Type Detection](https://mlsanity.com/paper/2408.16173/) (2024)
- [WeShap: Weak Supervision Source Evaluation with Shapley Values](https://mlsanity.com/paper/2406.11010/) (2024)

## Abstract (arXiv)

Programmatic weak supervision methodologies facilitate the expedited labeling of extensive datasets through the use of label functions (LFs) that encapsulate heuristic data sources. Nonetheless, the creation of precise LFs necessitates domain expertise and substantial endeavors. Recent advances in pre-trained language models (PLMs) have exhibited substantial potential across diverse tasks. However, the capacity of PLMs to autonomously formulate accurate LFs remains an underexplored domain. In this research, we address this gap by introducing DataSculpt, an interactive framework that harnesses PLMs for the automated generation of LFs. Within DataSculpt, we incorporate an array of prompting techniques, instance selection strategies, and LF filtration methods to explore the expansive design landscape. Ultimately, we conduct a thorough assessment of DataSculpt's performance on 12 real-world datasets, encompassing a range of tasks. This evaluation unveils both the strengths and limitations of contemporary PLMs in LF design.