# Interpolation between Residual and Non-Residual Networks

arXiv: https://arxiv.org/abs/2006.05749 · HTML: https://mlsanity.com/paper/2006.05749/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 6 of 6)

- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)
- [Neural Variable-Order Fractional Differential Equation Networks](https://mlsanity.com/paper/2503.16207/) (2025)
- [Diffusion Mechanism in Residual Neural Network: Theory and Applications](https://mlsanity.com/paper/2105.03155/) (2021)
- [Auto-Compressing Networks](https://mlsanity.com/paper/2506.09714/) (2025)
- [Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics](https://mlsanity.com/paper/2306.01435/) (2023)
- [Explanation-based Counterfactual Retraining(XCR): A Calibration Method for Black-box Models](https://mlsanity.com/paper/2206.11126/) (2022)

## Abstract (arXiv)

Although ordinary differential equations (ODEs) provide insights for designing network architectures, its relationship with the non-residual convolutional neural networks (CNNs) is still unclear. In this paper, we present a novel ODE model by adding a damping term. It can be shown that the proposed model can recover both a ResNet and a CNN by adjusting an interpolation coefficient. Therefore, the damped ODE model provides a unified framework for the interpretation of residual and non-residual networks. The Lyapunov analysis reveals better stability of the proposed model, and thus yields robustness improvement of the learned networks. Experiments on a number of image classification benchmarks show that the proposed model substantially improves the accuracy of ResNet and ResNeXt over the perturbed inputs from both stochastic noise and adversarial attack methods. Moreover, the loss landscape analysis demonstrates the improved robustness of our method along the attack direction.