# Residual Networks of Residual Networks: Multilevel Residual Networks

arXiv: https://arxiv.org/abs/1608.02908 · HTML: https://mlsanity.com/paper/1608.02908/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2016.

## Built on by (most cited first, 13 of 13)

- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2203.08921/) (2022)
- [ActiveMatch: End-to-end Semi-supervised Active Representation Learning](https://mlsanity.com/paper/2110.02521/) (2021)
- [Machine-Learning-Assisted and Real-Time-Feedback-Controlled Growth of InAs/GaAs Quantum Dots](https://mlsanity.com/paper/2306.12898/) (2023)
- [Approximation properties of Residual Neural Networks for Kolmogorov PDEs](https://mlsanity.com/paper/2111.00215/) (2021)
- [A Robust Deep Networks based Multi-Object MultiCamera Tracking System for City Scale Traffic](https://mlsanity.com/paper/2505.00534/) (2025)
- [Neural Generalized Ordinary Differential Equations with Layer-varying Parameters](https://mlsanity.com/paper/2209.10633/) (2022)
- [Artificial Human Intelligence: The role of Humans in the Development of Next Generation AI](https://mlsanity.com/paper/2409.16001/) (2024)
- [Spiking Neural Network Feature Discrimination Boosts Modality Fusion](https://mlsanity.com/paper/2502.10423/) (2025)
- [A Framework for 3D Tracking of Frontal Dynamic Objects in Autonomous Cars](https://mlsanity.com/paper/2103.13430/) (2021)
- [Air Quality Forecasting Using Machine Learning: A Global perspective with Relevance to Low-Resource Settings](https://mlsanity.com/paper/2401.04369/) (2024)
- [MBInception: A new Multi-Block Inception Model for Enhancing Image Processing Efficiency](https://mlsanity.com/paper/2412.13703/) (2024)
- [Towards Performance-Enhanced Model-Contrastive Federated Learning using Historical Information in Heterogeneous Scenarios](https://mlsanity.com/paper/2602.11945/) (2026)

## Abstract (arXiv)

A residual-networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual-network architecture, Residual networks of Residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds level-wise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4+SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100 and SVHN, with test errors 3.77%, 19.73% and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared to ResNets on ImageNet data set.