# CODA: Constructivism Learning for Instance-Dependent Dropout Architecture Construction

arXiv: https://arxiv.org/abs/2106.08444 · HTML: https://mlsanity.com/paper/2106.08444/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 8 of 8)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Variational Dropout Sparsifies Deep Neural Networks](https://mlsanity.com/paper/1701.05369/) (2017)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Generalized Dropout](https://mlsanity.com/paper/1611.06791/) (2016)
- [Blockout: Dynamic Model Selection for Hierarchical Deep Networks](https://mlsanity.com/paper/1512.05246/) (2015)
- [Improved Dropout for Shallow and Deep Learning](https://mlsanity.com/paper/1602.02220/) (2016)
- [A Bayesian encourages dropout](https://mlsanity.com/paper/1412.7003/) (2014)

## Abstract (arXiv)

Dropout is attracting intensive research interest in deep learning as an efficient approach to prevent overfitting. Recently incorporating structural information when deciding which units to drop out produced promising results comparing to methods that ignore the structural information. However, a major issue of the existing work is that it failed to differentiate among instances when constructing the dropout architecture. This can be a significant deficiency for many applications. To solve this issue, we propose Constructivism learning for instance-dependent Dropout Architecture (CODA), which is inspired from a philosophical theory, constructivism learning. Specially, based on the theory we have designed a better drop out technique, Uniform Process Mixture Models, using a Bayesian nonparametric method Uniform process. We have evaluated our proposed method on 5 real-world datasets and compared the performance with other state-of-the-art dropout techniques. The experimental results demonstrated the effectiveness of CODA.