# Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis

arXiv: https://arxiv.org/abs/2111.15186 · HTML: https://mlsanity.com/paper/2111.15186/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 20 of 20)

- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [Neuro-Symbolic Program Synthesis](https://mlsanity.com/paper/1611.01855/) (2016)
- [Learning to Infer Graphics Programs from Hand-Drawn Images](https://mlsanity.com/paper/1707.09627/) (2017)
- [Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling](https://mlsanity.com/paper/2012.06046/) (2020)
- [Self-Training with Weak Supervision](https://mlsanity.com/paper/2104.05514/) (2021)
- [Generating Multi-Agent Trajectories using Programmatic Weak Supervision](https://mlsanity.com/paper/1803.07612/) (2018)
- [HOUDINI: Lifelong Learning as Program Synthesis](https://mlsanity.com/paper/1804.00218/) (2018)
- [Cross-Modal Data Programming Enables Rapid Medical Machine Learning](https://mlsanity.com/paper/1903.11101/) (2019)
- [Learning Differentiable Programs with Admissible Neural Heuristics](https://mlsanity.com/paper/2007.12101/) (2020)
- [The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions](https://mlsanity.com/paper/2104.02710/) (2021)
- [Game Plan: What AI can do for Football, and What Football can do for AI](https://mlsanity.com/paper/2011.09192/) (2020)
- [Task Programming: Learning Data Efficient Behavior Representations](https://mlsanity.com/paper/2011.13917/) (2020)
- [Mapping the stereotyped behaviour of freely-moving fruit flies](https://mlsanity.com/paper/1310.4249/) (2013)
- [Active WeaSuL: Improving Weak Supervision with Active Learning](https://mlsanity.com/paper/2104.14847/) (2021)
- [Web Question Answering with Neurosymbolic Program Synthesis](https://mlsanity.com/paper/2104.07162/) (2021)

## Built on by (most cited first, 6 of 6)

- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2308.12595/) (2023)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [Neurosymbolic Programming for Science](https://mlsanity.com/paper/2210.05050/) (2022)
- [Coneheads: Hierarchy Aware Attention](https://mlsanity.com/paper/2306.00392/) (2023)
- [A Benchmark Generative Probabilistic Model for Weak Supervised Learning](https://mlsanity.com/paper/2303.17841/) (2023)

## Abstract (arXiv)

Obtaining annotations for large training sets is expensive, especially in settings where domain knowledge is required, such as behavior analysis. Weak supervision has been studied to reduce annotation costs by using weak labels from task-specific labeling functions (LFs) to augment ground truth labels. However, domain experts still need to hand-craft different LFs for different tasks, limiting scalability. To reduce expert effort, we present AutoSWAP: a framework for automatically synthesizing data-efficient task-level LFs. The key to our approach is to efficiently represent expert knowledge in a reusable domain-specific language and more general domain-level LFs, with which we use state-of-the-art program synthesis techniques and a small labeled dataset to generate task-level LFs. Additionally, we propose a novel structural diversity cost that allows for efficient synthesis of diverse sets of LFs, further improving AutoSWAP's performance. We evaluate AutoSWAP in three behavior analysis domains and demonstrate that AutoSWAP outperforms existing approaches using only a fraction of the data. Our results suggest that AutoSWAP is an effective way to automatically generate LFs that can significantly reduce expert effort for behavior analysis.