# Correcting Model Bias with Sparse Implicit Processes

arXiv: https://arxiv.org/abs/2207.10673 · HTML: https://mlsanity.com/paper/2207.10673/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 10 of 10)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks](https://mlsanity.com/paper/1701.04722/) (2017)
- [Generalized Variational Inference: Three arguments for deriving new Posteriors](https://mlsanity.com/paper/1904.02063/) (2019)
- [Variational Implicit Processes](https://mlsanity.com/paper/1806.02390/) (2018)
- [Black-box α-divergence Minimization](https://mlsanity.com/paper/1511.03243/) (2015)
- [Function-space Inference with Sparse Implicit Processes](https://mlsanity.com/paper/2110.07618/) (2021)
- [Wide Mean-Field Variational Bayesian Neural Networks Ignore the Data](https://mlsanity.com/paper/2106.07052/) (2021)
- [Adversarial α-divergence Minimization for Bayesian Approximate Inference](https://mlsanity.com/paper/1909.06945/) (2019)

## Built on by (most cited first, 1 of 1)

- [R'enyi Neural Processes](https://mlsanity.com/paper/2405.15991/) (2024)

## Abstract (arXiv)

Model selection in machine learning (ML) is a crucial part of the Bayesian learning procedure. Model choice may impose strong biases on the resulting predictions, which can hinder the performance of methods such as Bayesian neural networks and neural samplers. On the other hand, newly proposed approaches for Bayesian ML exploit features of approximate inference in function space with implicit stochastic processes (a generalization of Gaussian processes). The approach of Sparse Implicit Processes (SIP) is particularly successful in this regard, since it is fully trainable and achieves flexible predictions. Here, we expand on the original experiments to show that SIP is capable of correcting model bias when the data generating mechanism differs strongly from the one implied by the model. We use synthetic datasets to show that SIP is capable of providing predictive distributions that reflect the data better than the exact predictions of the initial, but wrongly assumed model.