# Learning from Uncertain Data: From Possible Worlds to Possible Models

arXiv: https://arxiv.org/abs/2405.18549 · HTML: https://mlsanity.com/paper/2405.18549/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 82 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 30 of 35)

- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning](https://mlsanity.com/paper/1804.00308/) (2018)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Distributionally Robust Logistic Regression](https://mlsanity.com/paper/1509.09259/) (2015)
- [Certified Robustness to Label-Flipping Attacks via Randomized Smoothing](https://mlsanity.com/paper/2002.03018/) (2020)
- [Recent Advances in Algorithmic High-Dimensional Robust Statistics](https://mlsanity.com/paper/1911.05911/) (2019)
- [Intrinsic Certified Robustness of Bagging against Data Poisoning Attacks](https://mlsanity.com/paper/2008.04495/) (2020)
- [Manipulating SGD with Data Ordering Attacks](https://mlsanity.com/paper/2104.09667/) (2021)
- [Label Sanitization against Label Flipping Poisoning Attacks](https://mlsanity.com/paper/1803.00992/) (2018)
- [Learning Certified Individually Fair Representations](https://mlsanity.com/paper/2002.10312/) (2020)
- [All You Need is a Good Functional Prior for Bayesian Deep Learning](https://mlsanity.com/paper/2011.12829/) (2020)
- [Predictive Inference Is Free with the Jackknife+-after-Bootstrap](https://mlsanity.com/paper/2002.09025/) (2020)
- [GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization](https://mlsanity.com/paper/2107.02425/) (2021)
- [Valid prediction intervals for regression problems](https://mlsanity.com/paper/2107.00363/) (2021)
- [Training Set Debugging Using Trusted Items](https://mlsanity.com/paper/1801.08019/) (2018)
- [Verification of Neural-Network Control Systems by Integrating Taylor Models and Zonotopes](https://mlsanity.com/paper/2112.09197/) (2021)
- [Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning](https://mlsanity.com/paper/2207.02598/) (2022)
- [Conformal Prediction with Missing Values](https://mlsanity.com/paper/2306.02732/) (2023)
- [Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions](https://mlsanity.com/paper/2005.05117/) (2020)
- [Hyperparameter Optimization Is Deceiving Us, and How to Stop It](https://mlsanity.com/paper/2102.03034/) (2021)
- [Robustness Certification of Generative Models](https://mlsanity.com/paper/2004.14756/) (2020)
- [The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictions](https://mlsanity.com/paper/2304.10655/) (2023)
- [Sparse Polynomial Zonotopes: A Novel Set Representation for Reachability Analysis](https://mlsanity.com/paper/1901.01780/) (2019)
- [What to Expect of Classifiers? Reasoning about Logistic Regression with Missing Features](https://mlsanity.com/paper/1903.01620/) (2019)
- [Prior choice affects ability of Bayesian neural networks to identify unknowns](https://mlsanity.com/paper/2005.04987/) (2020)
- [Data Poisoning Attacks on Regression Learning and Corresponding Defenses](https://mlsanity.com/paper/2009.07008/) (2020)
- [Data-Driven Reachability Analysis Using Matrix Zonotopes](https://mlsanity.com/paper/2011.08472/) (2020)
- [Consistent Range Approximation for Fair Predictive Modeling](https://mlsanity.com/paper/2212.10839/) (2022)

## Abstract (arXiv)

We introduce an efficient method for learning linear models from uncertain data, where uncertainty is represented as a set of possible variations in the data, leading to predictive multiplicity. Our approach leverages abstract interpretation and zonotopes, a type of convex polytope, to compactly represent these dataset variations, enabling the symbolic execution of gradient descent on all possible worlds simultaneously. We develop techniques to ensure that this process converges to a fixed point and derive closed-form solutions for this fixed point. Our method provides sound over-approximations of all possible optimal models and viable prediction ranges. We demonstrate the effectiveness of our approach through theoretical and empirical analysis, highlighting its potential to reason about model and prediction uncertainty due to data quality issues in training data.