# Blockout: Dynamic Model Selection for Hierarchical Deep Networks

arXiv: https://arxiv.org/abs/1512.05246 · HTML: https://mlsanity.com/paper/1512.05246/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2015.

## Built on by (most cited first, 11 of 11)

- [Neural Networks are Decision Trees](https://mlsanity.com/paper/2210.05189/) (2022)
- [Data Augmented 3D Semantic Scene Completion with 2D Segmentation Priors](https://mlsanity.com/paper/2111.13309/) (2021)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [Learning Label Hierarchy with Supervised Contrastive Learning](https://mlsanity.com/paper/2402.00232/) (2024)
- [Transition Matrix Representation of Trees with Transposed Convolutions](https://mlsanity.com/paper/2202.10677/) (2022)
- [Deep P-Spline: Theory, Fast Tuning, and Application](https://mlsanity.com/paper/2501.01376/) (2025)
- [CODA: Constructivism Learning for Instance-Dependent Dropout Architecture Construction](https://mlsanity.com/paper/2106.08444/) (2021)
- [LEURN: Learning Explainable Univariate Rules with Neural Networks](https://mlsanity.com/paper/2303.14937/) (2023)
- [Conditional Information Gain Trellis](https://mlsanity.com/paper/2402.08345/) (2024)
- [MaskAnyNet: Rethinking Masked Image Regions as Valuable Information in Supervised Learning](https://mlsanity.com/paper/2511.12480/) (2025)
- [Concept Flow Models: Anchoring Concept-Based Reasoning with Hierarchical Bottlenecks](https://mlsanity.com/paper/2606.19489/) (2026)

## Abstract (arXiv)

Most deep architectures for image classification--even those that are trained to classify a large number of diverse categories--learn shared image representations with a single model. Intuitively, however, categories that are more similar should share more information than those that are very different. While hierarchical deep networks address this problem by learning separate features for subsets of related categories, current implementations require simplified models using fixed architectures specified via heuristic clustering methods. Instead, we propose Blockout, a method for regularization and model selection that simultaneously learns both the model architecture and parameters. A generalization of Dropout, our approach gives a novel parametrization of hierarchical architectures that allows for structure learning via back-propagation. To demonstrate its utility, we evaluate Blockout on the CIFAR and ImageNet datasets, demonstrating improved classification accuracy, better regularization performance, faster training, and the clear emergence of hierarchical network structures.