# Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision

arXiv: https://arxiv.org/abs/1808.08485 · HTML: https://mlsanity.com/paper/1808.08485/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2018.

## Built on by (most cited first, 9 of 9)

- [End-to-End Weak Supervision](https://mlsanity.com/paper/2107.02233/) (2021)
- [Learning Logic Rules for Document-level Relation Extraction](https://mlsanity.com/paper/2111.05407/) (2021)
- [Transformers and the representation of biomedical background knowledge](https://mlsanity.com/paper/2202.02432/) (2022)
- [Pareto Optimal Learning for Estimating Large Language Model Errors](https://mlsanity.com/paper/2306.16564/) (2023)
- [Modular Self-Supervision for Document-Level Relation Extraction](https://mlsanity.com/paper/2109.05362/) (2021)
- [Principled and Efficient Motif Finding for Structure Learning of Lifted Graphical Models](https://mlsanity.com/paper/2302.04599/) (2023)
- [Towards Structuring Real-World Data at Scale: Deep Learning for Extracting Key Oncology Information from Clinical Text with Patient-Level Supervision](https://mlsanity.com/paper/2203.10442/) (2022)
- [Parallel Neurosymbolic Integration with Concordia](https://mlsanity.com/paper/2306.00480/) (2023)
- [Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning](https://mlsanity.com/paper/2107.12591/) (2021)

## Abstract (arXiv)

Deep learning has emerged as a versatile tool for a wide range of NLP tasks, due to its superior capacity in representation learning. But its applicability is limited by the reliance on annotated examples, which are difficult to produce at scale. Indirect supervision has emerged as a promising direction to address this bottleneck, either by introducing labeling functions to automatically generate noisy examples from unlabeled text, or by imposing constraints over interdependent label decisions. A plethora of methods have been proposed, each with respective strengths and limitations. Probabilistic logic offers a unifying language to represent indirect supervision, but end-to-end modeling with probabilistic logic is often infeasible due to intractable inference and learning. In this paper, we propose deep probabilistic logic (DPL) as a general framework for indirect supervision, by composing probabilistic logic with deep learning. DPL models label decisions as latent variables, represents prior knowledge on their relations using weighted first-order logical formulas, and alternates between learning a deep neural network for the end task and refining uncertain formula weights for indirect supervision, using variational EM. This framework subsumes prior indirect supervision methods as special cases, and enables novel combination via infusion of rich domain and linguistic knowledge. Experiments on biomedical machine reading demonstrate the promise of this approach.