# Learning Identity Mappings with Residual Gates

arXiv: https://arxiv.org/abs/1611.01260 · HTML: https://mlsanity.com/paper/1611.01260/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2016.

## Built on by (most cited first, 6 of 6)

- [RegNet: Self-Regulated Network for Image Classification](https://mlsanity.com/paper/2101.00590/) (2021)
- [LAuReL: Learned Augmented Residual Layer](https://mlsanity.com/paper/2411.07501/) (2024)
- [Auto-Compressing Networks](https://mlsanity.com/paper/2506.09714/) (2025)
- [Towards Objective and Unbiased Decision Assessments with LLM-Enhanced Hierarchical Attention Networks](https://mlsanity.com/paper/2411.08504/) (2024)
- [A Multi-Branch Hierarchy-Aware Framework for Heterogeneous Audio Classification](https://mlsanity.com/paper/2607.01974/) (2026)
- [Do ResNets Route? Sparse Interaction Experts in Residual Networks](https://mlsanity.com/paper/2610.02907/) (2026)

## Abstract (arXiv)

We propose a new layer design by adding a linear gating mechanism to shortcut connections. By using a scalar parameter to control each gate, we provide a way to learn identity mappings by optimizing only one parameter. We build upon the motivation behind Residual Networks, where a layer is reformulated in order to make learning identity mappings less problematic to the optimizer. The augmentation introduces only one extra parameter per layer, and provides easier optimization by making degeneration into identity mappings simpler. We propose a new model, the Gated Residual Network, which is the result when augmenting Residual Networks. Experimental results show that augmenting layers provides better optimization, increased performance, and more layer independence. We evaluate our method on MNIST using fully-connected networks, showing empirical indications that our augmentation facilitates the optimization of deep models, and that it provides high tolerance to full layer removal: the model retains over 90% of its performance even after half of its layers have been randomly removed. We also evaluate our model on CIFAR-10 and CIFAR-100 using Wide Gated ResNets, achieving 3.65% and 18.27% error, respectively.