# Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1512.05830 · HTML: https://mlsanity.com/paper/1512.05830/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2015.

## Built on by (most cited first, 30 of 115)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Improving Calibration for Long-Tailed Recognition](https://mlsanity.com/paper/2104.00466/) (2021)
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](https://mlsanity.com/paper/2103.16370/) (2021)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [Probabilistic two-stage detection](https://mlsanity.com/paper/2103.07461/) (2021)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [Simple multi-dataset detection](https://mlsanity.com/paper/2102.13086/) (2021)
- [A Survey on Long-Tailed Visual Recognition](https://mlsanity.com/paper/2205.13775/) (2022)
- [Self Supervision to Distillation for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2109.04075/) (2021)
- [Distilling Virtual Examples for Long-tailed Recognition](https://mlsanity.com/paper/2103.15042/) (2021)
- [Adversarial Robustness under Long-Tailed Distribution](https://mlsanity.com/paper/2104.02703/) (2021)
- [AutoBalance: Optimized Loss Functions for Imbalanced Data](https://mlsanity.com/paper/2201.01212/) (2022)
- [Self-Damaging Contrastive Learning](https://mlsanity.com/paper/2106.02990/) (2021)
- [Generalized Parametric Contrastive Learning](https://mlsanity.com/paper/2209.12400/) (2022)
- [Towards Calibrated Model for Long-Tailed Visual Recognition from Prior Perspective](https://mlsanity.com/paper/2111.03874/) (2021)
- [Faster Meta Update Strategy for Noise-Robust Deep Learning](https://mlsanity.com/paper/2104.15092/) (2021)
- [Robust Long-Tailed Learning under Label Noise](https://mlsanity.com/paper/2108.11569/) (2021)
- [Feedback-guided Data Synthesis for Imbalanced Classification](https://mlsanity.com/paper/2310.00158/) (2023)
- [FEND: A Future Enhanced Distribution-Aware Contrastive Learning Framework for Long-tail Trajectory Prediction](https://mlsanity.com/paper/2303.16574/) (2023)
- [Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling](https://mlsanity.com/paper/2111.01004/) (2021)
- [Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection](https://mlsanity.com/paper/2104.05702/) (2021)
- [Balancing Logit Variation for Long-tailed Semantic Segmentation](https://mlsanity.com/paper/2306.02061/) (2023)
- [Subclass-balancing Contrastive Learning for Long-tailed Recognition](https://mlsanity.com/paper/2306.15925/) (2023)
- [Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasets](https://mlsanity.com/paper/2206.08802/) (2022)
- [Constructing Balance from Imbalance for Long-tailed Image Recognition](https://mlsanity.com/paper/2208.02567/) (2022)
- [Iterative Scene Graph Generation](https://mlsanity.com/paper/2207.13440/) (2022)
- [Open Long-Tailed Recognition in a Dynamic World](https://mlsanity.com/paper/2208.08349/) (2022)
- [A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing](https://mlsanity.com/paper/2210.04675/) (2022)
- [When Noisy Labels Meet Long Tail Dilemmas: A Representation Calibration Method](https://mlsanity.com/paper/2211.10955/) (2022)
- [Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification](https://mlsanity.com/paper/2112.14380/) (2021)

## Abstract (arXiv)

Learning deeper convolutional neural networks becomes a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be gained by simply stacking more layers. In this paper, we consider the issue from an information theoretical perspective, and propose a novel method Relay Backpropagation, that encourages the propagation of effective information through the network in training stage. By virtue of the method, we achieved the first place in ILSVRC 2015 Scene Classification Challenge. Extensive experiments on two challenging large scale datasets demonstrate the effectiveness of our method is not restricted to a specific dataset or network architecture. Our models will be available to the research community later.