# Deep Convolutional Neural Networks with Merge-and-Run Mappings

arXiv: https://arxiv.org/abs/1611.07718 · HTML: https://mlsanity.com/paper/1611.07718/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2016.

## Built on by (most cited first, 4 of 4)

- [On the Connection between Local Attention and Dynamic Depth-wise Convolution](https://mlsanity.com/paper/2106.04263/) (2021)
- [DFTR: Depth-supervised Fusion Transformer for Salient Object Detection](https://mlsanity.com/paper/2203.06429/) (2022)
- [Deep Idempotent Network for Efficient Single Image Blind Deblurring](https://mlsanity.com/paper/2210.07122/) (2022)
- [A residual dense vision transformer for medical image super-resolution with segmentation-based perceptual loss fine-tuning](https://mlsanity.com/paper/2302.11184/) (2023)

## Abstract (arXiv)

A deep residual network, built by stacking a sequence of residual blocks, is easy to train, because identity mappings skip residual branches and thus improve information flow. To further reduce the training difficulty, we present a simple network architecture, deep merge-and-run neural networks. The novelty lies in a modularized building block, merge-and-run block, which assembles residual branches in parallel through a merge-and-run mapping: Average the inputs of these residual branches (Merge), and add the average to the output of each residual branch as the input of the subsequent residual branch (Run), respectively. We show that the merge-and-run mapping is a linear idempotent function in which the transformation matrix is idempotent, and thus improves information flow, making training easy. In comparison to residual networks, our networks enjoy compelling advantages: they contain much shorter paths, and the width, i.e., the number of channels, is increased. We evaluate the performance on the standard recognition tasks. Our approach demonstrates consistent improvements over ResNets with the comparable setup, and achieves competitive results (e.g., 3.57% testing error on CIFAR-10, 19.00% on CIFAR-100, 1.51% on SVHN).