# DropBlock: A regularization method for convolutional networks

arXiv: https://arxiv.org/abs/1810.12890 · HTML: https://mlsanity.com/paper/1810.12890/

## Key facts

- Cited by 262 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2018.

## Built on by (most cited first, 30 of 262)

- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Masked World Models for Visual Control](https://mlsanity.com/paper/2206.14244/) (2022)
- [Revisiting ResNets: Improved Training and Scaling Strategies](https://mlsanity.com/paper/2103.07579/) (2021)
- [Masked Discrimination for Self-Supervised Learning on Point Clouds](https://mlsanity.com/paper/2203.11183/) (2022)
- [A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS](https://mlsanity.com/paper/2304.00501/) (2023)
- [Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization](https://mlsanity.com/paper/2203.07740/) (2022)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [What to Hide from Your Students: Attention-Guided Masked Image Modeling](https://mlsanity.com/paper/2203.12719/) (2022)
- [Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2111.14213/) (2021)
- [Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation](https://mlsanity.com/paper/2103.14962/) (2021)
- [Adversarially Adaptive Normalization for Single Domain Generalization](https://mlsanity.com/paper/2106.01899/) (2021)
- [TransFER: Learning Relation-aware Facial Expression Representations with Transformers](https://mlsanity.com/paper/2108.11116/) (2021)
- [PP-YOLOE: An evolved version of YOLO](https://mlsanity.com/paper/2203.16250/) (2022)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [MixStyle Neural Networks for Domain Generalization and Adaptation](https://mlsanity.com/paper/2107.02053/) (2021)
- [Regularization Strategy for Point Cloud via Rigidly Mixed Sample](https://mlsanity.com/paper/2102.01929/) (2021)
- [Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers](https://mlsanity.com/paper/2303.01610/) (2023)
- [No One Representation to Rule Them All: Overlapping Features of Training Methods](https://mlsanity.com/paper/2110.12899/) (2021)
- [Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning](https://mlsanity.com/paper/2103.01315/) (2021)
- [Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks](https://mlsanity.com/paper/2201.03299/) (2022)
- [Evaluation of post-hoc interpretability methods in time-series classification](https://mlsanity.com/paper/2202.05656/) (2022)
- [Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification](https://mlsanity.com/paper/2204.04567/) (2022)
- [Scaling Laws for Downstream Task Performance of Large Language Models](https://mlsanity.com/paper/2402.04177/) (2024)
- [Rethinking "Batch" in BatchNorm](https://mlsanity.com/paper/2105.07576/) (2021)
- [Rethinking Pseudo Labels for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.00168/) (2021)
- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [Optimized U-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2110.03352/) (2021)
- [Adaptive Transformers for Robust Few-shot Cross-domain Face Anti-spoofing](https://mlsanity.com/paper/2203.12175/) (2022)
- [AutoDropout: Learning Dropout Patterns to Regularize Deep Networks](https://mlsanity.com/paper/2101.01761/) (2021)

## Abstract (arXiv)

Deep neural networks often work well when they are over-parameterized and trained with a massive amount of noise and regularization, such as weight decay and dropout. Although dropout is widely used as a regularization technique for fully connected layers, it is often less effective for convolutional layers. This lack of success of dropout for convolutional layers is perhaps due to the fact that activation units in convolutional layers are spatially correlated so information can still flow through convolutional networks despite dropout. Thus a structured form of dropout is needed to regularize convolutional networks. In this paper, we introduce DropBlock, a form of structured dropout, where units in a contiguous region of a feature map are dropped together. We found that applying DropbBlock in skip connections in addition to the convolution layers increases the accuracy. Also, gradually increasing number of dropped units during training leads to better accuracy and more robust to hyperparameter choices. Extensive experiments show that DropBlock works better than dropout in regularizing convolutional networks. On ImageNet classification, ResNet-50 architecture with DropBlock achieves 78.13% accuracy, which is more than 1.6% improvement on the baseline. On COCO detection, DropBlock improves Average Precision of RetinaNet from 36.8% to 38.4%.