# Towards Robust ResNet: A Small Step but A Giant Leap

arXiv: https://arxiv.org/abs/1902.10887 · HTML: https://mlsanity.com/paper/1902.10887/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2019.

## Built on by (most cited first, 15 of 15)

- [Noisy Recurrent Neural Networks](https://mlsanity.com/paper/2102.04877/) (2021)
- [Scaling ResNets in the Large-depth Regime](https://mlsanity.com/paper/2206.06929/) (2022)
- [Neural equilibria for long-term prediction of nonlinear conservation laws](https://mlsanity.com/paper/2501.06933/) (2025)
- [Double Visual Defense: Adversarial Pre-training and Instruction Tuning for Improving Vision-Language Model Robustness](https://mlsanity.com/paper/2501.09446/) (2025)
- [Field theory for optimal signal propagation in ResNets](https://mlsanity.com/paper/2305.07715/) (2023)
- [Module-wise Training of Neural Networks via the Minimizing Movement Scheme](https://mlsanity.com/paper/2309.17357/) (2023)
- [A Differential Geometry Perspective on Orthogonal Recurrent Models](https://mlsanity.com/paper/2102.09589/) (2021)
- [IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method](https://mlsanity.com/paper/2505.06889/) (2025)
- [Zero Stability Well Predicts Performance of Convolutional Neural Networks](https://mlsanity.com/paper/2206.13100/) (2022)
- [Block-wise Training of Residual Networks via the Minimizing Movement Scheme](https://mlsanity.com/paper/2210.00949/) (2022)
- [Adversarial Sample Detection Through Neural Network Transport Dynamics](https://mlsanity.com/paper/2306.04252/) (2023)
- [Generalization by design: Shortcuts to Generalization in Deep Learning](https://mlsanity.com/paper/2107.02253/) (2021)
- [Deep Neural Networks and PIDE discretizations](https://mlsanity.com/paper/2108.02430/) (2021)
- [Randomness Helps Rigor: A Probabilistic Learning Rate Scheduler Bridging Theory and Deep Learning Practice](https://mlsanity.com/paper/2407.07613/) (2024)
- [Geometry Aware Meta-Learning Neural Network for Joint Phase and Precoder Optimization in RIS](https://mlsanity.com/paper/2409.11270/) (2024)

## Abstract (arXiv)

This paper presents a simple yet principled approach to boosting the robustness of the residual network (ResNet) that is motivated by the dynamical system perspective. Namely, a deep neural network can be interpreted using a partial differential equation, which naturally inspires us to characterize ResNet by an explicit Euler method. Our analytical studies reveal that the step factor h in the Euler method is able to control the robustness of ResNet in both its training and generalization. Specifically, we prove that a small step factor h can benefit the training robustness for back-propagation; from the view of forward-propagation, a small h can aid in the robustness of the model generalization. A comprehensive empirical evaluation on both vision CIFAR-10 and text AG-NEWS datasets confirms that a small h aids both the training and generalization robustness.