# Incidental Supervision: Moving beyond Supervised Learning

arXiv: https://arxiv.org/abs/2005.12339 · HTML: https://mlsanity.com/paper/2005.12339/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2020.

## Built on by (most cited first, 13 of 13)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts](https://mlsanity.com/paper/2210.16865/) (2022)
- [ROCK: Causal Inference Principles for Reasoning about Commonsense Causality](https://mlsanity.com/paper/2202.00436/) (2022)
- [Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?](https://mlsanity.com/paper/2212.10784/) (2022)
- [Learning Constraints and Descriptive Segmentation for Subevent Detection](https://mlsanity.com/paper/2109.06316/) (2021)
- [Toward a `Standard Model' of Machine Learning](https://mlsanity.com/paper/2108.07783/) (2021)
- [Dense Retrieval as Indirect Supervision for Large-space Decision Making](https://mlsanity.com/paper/2310.18619/) (2023)
- [SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning](https://mlsanity.com/paper/2409.17755/) (2024)
- [Diagnosing the Performance Trade-off in Moral Alignment: A Case Study on Gender Stereotypes](https://mlsanity.com/paper/2509.21456/) (2025)
- [The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck](https://mlsanity.com/paper/2211.07959/) (2022)
- [WHERE and WHICH: Iterative Debate for Biomedical Synthetic Data Augmentation](https://mlsanity.com/paper/2503.23673/) (2025)
- [Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning](https://mlsanity.com/paper/2107.12591/) (2021)
- [Cross-Lingual Speaker Identification Using Distant Supervision](https://mlsanity.com/paper/2210.05780/) (2022)

## Abstract (arXiv)

Machine Learning and Inference methods have become ubiquitous in our attempt to induce more abstract representations of natural language text, visual scenes, and other messy, naturally occurring data, and support decisions that depend on it. However, learning models for these tasks is difficult partly because generating the necessary supervision signals for it is costly and does not scale. This paper describes several learning paradigms that are designed to alleviate the supervision bottleneck. It will illustrate their benefit in the context of multiple problems, all pertaining to inducing various levels of semantic representations from text.