# Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregation

arXiv: https://arxiv.org/abs/2110.11852 · HTML: https://mlsanity.com/paper/2110.11852/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 62 references that are in the map.
- First posted on arXiv 22 Oct 2021.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)

## Built on by (most cited first, 13 of 13)

- [Dense Learning based Semi-Supervised Object Detection](https://mlsanity.com/paper/2204.07300/) (2022)
- [Cross-Layer Retrospective Retrieving via Layer Attention](https://mlsanity.com/paper/2302.03985/) (2023)
- [Strengthening Layer Interaction via Dynamic Layer Attention](https://mlsanity.com/paper/2406.13392/) (2024)
- [Understanding Self-attention Mechanism via Dynamical System Perspective](https://mlsanity.com/paper/2308.09939/) (2023)
- [Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models](https://mlsanity.com/paper/2505.12343/) (2025)
- [A Generic Shared Attention Mechanism for Various Backbone Neural Networks](https://mlsanity.com/paper/2210.16101/) (2022)
- [TDAM: Top-Down Attention Module for Contextually Guided Feature Selection in CNNs](https://mlsanity.com/paper/2111.13470/) (2021)
- [HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2506.17587/) (2025)
- [Enhancing Layer Attention Efficiency through Pruning Redundant Retrievals](https://mlsanity.com/paper/2503.06473/) (2025)
- [Gramian Attention Heads are Strong yet Efficient Vision Learners](https://mlsanity.com/paper/2310.16483/) (2023)
- [BA-Net: Bridge Attention in Deep Neural Networks](https://mlsanity.com/paper/2410.07860/) (2024)
- [Enhancing Layer Interaction Using Key-Correlated Layer Attention](https://mlsanity.com/paper/2606.28405/) (2026)
- [Enhancing Oracle Bone Inscription Recognition via Multi-Scale Layer Attention](https://mlsanity.com/paper/2607.00057/) (2026)

## Abstract (arXiv)

This paper introduces a concept of layer aggregation to describe how information from previous layers can be reused to better extract features at the current layer. While DenseNet is a typical example of the layer aggregation mechanism, its redundancy has been commonly criticized in the literature. This motivates us to propose a very light-weighted module, called recurrent layer aggregation (RLA), by making use of the sequential structure of layers in a deep CNN. Our RLA module is compatible with many mainstream deep CNNs, including ResNets, Xception and MobileNetV2, and its effectiveness is verified by our extensive experiments on image classification, object detection and instance segmentation tasks. Specifically, improvements can be uniformly observed on CIFAR, ImageNet and MS COCO datasets, and the corresponding RLA-Nets can surprisingly boost the performances by 2-3% on the object detection task. This evidences the power of our RLA module in helping main CNNs better learn structural information in images.