# PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses

arXiv: https://arxiv.org/abs/2402.04987 · HTML: https://mlsanity.com/paper/2402.04987/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [On Learning from Label Proportions](https://mlsanity.com/paper/1402.5902/) (2014)
- [On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models](https://mlsanity.com/paper/2209.05310/) (2022)
- [Learning from Label Proportions by Learning with Label Noise](https://mlsanity.com/paper/2203.02496/) (2022)
- [Easy Learning from Label Proportions](https://mlsanity.com/paper/2302.03115/) (2023)
- [Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems](https://mlsanity.com/paper/2305.12102/) (2023)
- [Learning from Aggregated Data: Curated Bags versus Random Bags](https://mlsanity.com/paper/2305.09557/) (2023)
- [Goodbye Tracking? Impact of iOS App Tracking Transparency and Privacy Labels](https://mlsanity.com/paper/2204.03556/) (2022)
- [Learning Rate Schedules in the Presence of Distribution Shift](https://mlsanity.com/paper/2303.15634/) (2023)
- [LLP-Bench: A Large Scale Tabular Benchmark for Learning from Label Proportions](https://mlsanity.com/paper/2310.10096/) (2023)
- [Learning from Aggregate responses: Instance Level versus Bag Level Loss Functions](https://mlsanity.com/paper/2401.11081/) (2024)

## Built on by (most cited first, 2 of 2)

- [Optimistic Rates for Learning from Label Proportions](https://mlsanity.com/paper/2406.00487/) (2024)
- [Aggregating Data for Optimal and Private Learning](https://mlsanity.com/paper/2411.19045/) (2024)

## Abstract (arXiv)

This work studies algorithms for learning from aggregate responses. We focus on the construction of aggregation sets (called bags in the literature) for event-level loss functions. We prove for linear regression and generalized linear models (GLMs) that the optimal bagging problem reduces to one-dimensional size-constrained k-means clustering. Further, we theoretically quantify the advantage of using curated bags over random bags. We then propose the PriorBoost algorithm, which adaptively forms bags of samples that are increasingly homogeneous with respect to (unobserved) individual responses to improve model quality. We study label differential privacy for aggregate learning, and we also provide extensive experiments showing that PriorBoost regularly achieves optimal model quality for event-level predictions, in stark contrast to non-adaptive algorithms.