ScriptoriumWS: A Code Generation Assistant for Weak Supervision
Not on the map. Why?
Abstract (arXiv)
Weak supervision is a popular framework for overcoming the labeled data bottleneck: the need to obtain labels for training data. In weak supervision, multiple noisy-but-cheap sources are used to provide guesses of the label and are aggregated to produce high-quality pseudolabels. These sources are often expressed as small programs written by domain experts -- and so are expensive to obtain. Instead, we argue for using code-generation models to act as coding assistants for crafting weak supervision sources. We study prompting strategies to maximize the quality of the generated sources, settling on a multi-tier strategy that incorporates multiple types of information. We explore how to best combine hand-written and generated sources. Using these insights, we introduce ScriptoriumWS, a weak supervision system that, when compared to hand-crafted sources, maintains accuracy and greatly improves coverage.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 17 Feb 2025.
Builds on
27 of 35 references.
References · 27
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
- 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
- 2022A Systematic Evaluation of Large Language Models of Codecited by 360
- 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
- 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
- 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
15 more
- 2022Automatic Generation of Programming Exercises and Code Explanations using Large Language Modelscited by 124
- 2018Training Complex Models with Multi-Task Weak Supervisioncited by 106
- 2020Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methodscited by 98
- 2021WRENCH: A Comprehensive Benchmark for Weak Supervisioncited by 95
- 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 81
- 2018Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scalecited by 52
- 2020Interactive Weak Supervision: Learning Useful Heuristics for Data Labelingcited by 52
- 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 51
- 2020Denoising Multi-Source Weak Supervision for Neural Text Classificationcited by 42
- 2021Weakly Supervised Named Entity Tagging with Learnable Logical Rulescited by 21
- 2019GOGGLES: Automatic Image Labeling with Affinity Codingcited by 19
- 2021GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognitioncited by 12
- 2022Lifting Weak Supervision To Structured Predictioncited by 10
- 2022Classifying Unstructured Clinical Notes via Automatic Weak Supervisioncited by 7
- 2022AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labelscited by 7
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.