mlsanityData to 7 Oct 2026

Creating Training Sets via Weak Indirect Supervision

arXiv 2110.03484 ↗First posted 7 Oct 2021cs.LG stat.AP stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

Creating labeled training sets has become one of the major roadblocks in machine learning. To address this, recent Weak Supervision (WS) frameworks synthesize training labels from multiple potentially noisy supervision sources. However, existing frameworks are restricted to supervision sources that share the same output space as the target task. To extend the scope of usable sources, we formulate Weak Indirect Supervision (WIS), a new research problem for automatically synthesizing training labels based on indirect supervision sources that have different output label spaces. To overcome the challenge of mismatched output spaces, we develop a probabilistic modeling approach, PLRM, which uses user-provided label relations to model and leverage indirect supervision sources. Moreover, we provide a theoretically-principled test of the distinguishability of PLRM for unseen labels, along with a generalization bound. On both image and text classification tasks as well as an industrial advertising application, we demonstrate the advantages of PLRM by outperforming baselines by a margin of 2%-9%.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 26 of its 53 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

26 of 53 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
  5. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  6. 2018Training Complex Models with Multi-Task Weak Supervisioncited by 106
  7. 2020Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methodscited by 98
  8. 2021WRENCH: A Comprehensive Benchmark for Weak Supervisioncited by 95
  9. 2017Who Said What: Modeling Individual Labelers Improves Classificationcited by 77
  10. 2018Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scalecited by 52
  11. 2017Learning the Structure of Generative Models without Labeled Datacited by 50
  12. 2017Learning From Noisy Singly-labeled Datacited by 50
14 more
  1. 2018Generating Multi-Agent Trajectories using Programmatic Weak Supervisioncited by 46
  2. 2020Named Entity Recognition without Labelled Data: A Weak Supervision Approachcited by 46
  3. 2020Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphscited by 39
  4. 2020FrugalML: How to Use ML Prediction APIs More Accurately and Cheaplycited by 33
  5. 2019Cross-Modal Data Programming Enables Rapid Medical Machine Learningcited by 32
  6. 2020AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Typescited by 31
  7. 2019NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extractioncited by 24
  8. 2015LSHTC: A Benchmark for Large-Scale Text Classificationcited by 22
  9. 2020Ontology-driven weak supervision for clinical entity classification in electronic health recordscited by 18
  10. 2019Multi-Resolution Weak Supervision for Sequential Datacited by 17
  11. 2020Learnability with Indirect Supervision Signalscited by 7
  12. 2016Estimation from Indirect Supervision with Linear Momentscited by 6
  13. 2017Finer Grained Entity Typing with TypeNetcited by 3
  14. 2019Learning from Indirect Observationscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 7; 2023: 2; 2024: 5.

14 papers.

Sort:

By area:

  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  2. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  3. 2022A Survey on Programmatic Weak Supervisioncited by 66
  4. 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 51
  5. 2022Losses over Labels: Weakly Supervised Learning via Direct Loss Constructioncited by 11
  6. 2024Decoding the Narratives: Analyzing Personal Drug Experiences Shared on Redditcited by 9
  7. 2022Understanding Programmatic Weak Supervision via Source-aware Influence Functioncited by 8
  8. 2023How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Rankingcited by 8
  9. 2022Adaptive Ranking-based Sample Selection for Weakly Supervised Class-imbalanced Text Classificationcited by 7
  10. 2022Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervisioncited by 6
  11. 2024Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMscited by 3
  12. 2022Binary Classification with Positive Labeling Sourcescited by 2
2 more
  1. 2024Leveraging Large Language Models for Structure Learning in Prompted Weak Supervisioncited by 1
  2. 2024WeShap: Weak Supervision Source Evaluation with Shapley Valuescited by 1