# IMEXnet: A Forward Stable Deep Neural Network

arXiv: https://arxiv.org/abs/1903.02639 · HTML: https://mlsanity.com/paper/1903.02639/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2019.

## Built on by (most cited first, 22 of 22)

- [PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/2108.01938/) (2021)
- [Wavelet Convolutions for Large Receptive Fields](https://mlsanity.com/paper/2407.05848/) (2024)
- [Closed-form Continuous-time Neural Models](https://mlsanity.com/paper/2106.13898/) (2021)
- [Latent Space Data Assimilation by using Deep Learning](https://mlsanity.com/paper/2104.00430/) (2021)
- [Feature Transportation Improves Graph Neural Networks](https://mlsanity.com/paper/2307.16092/) (2023)
- [Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation using Compact Implicit Layers](https://mlsanity.com/paper/2306.17486/) (2023)
- [Connections between Numerical Algorithms for PDEs and Neural Networks](https://mlsanity.com/paper/2107.14742/) (2021)
- [Constrained Block Nonlinear Neural Dynamical Models](https://mlsanity.com/paper/2101.01864/) (2021)
- [Graph Neural Reaction Diffusion Models](https://mlsanity.com/paper/2406.10871/) (2024)
- [Advection Augmented Convolutional Neural Networks](https://mlsanity.com/paper/2406.19253/) (2024)
- [On the Stochastic Stability of Deep Markov Models](https://mlsanity.com/paper/2111.04601/) (2021)
- [Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations](https://mlsanity.com/paper/2109.00095/) (2021)
- [An Over Complete Deep Learning Method for Inverse Problems](https://mlsanity.com/paper/2402.04653/) (2024)
- [Semi-Implicit Neural Ordinary Differential Equations](https://mlsanity.com/paper/2412.11301/) (2024)
- [Zero Stability Well Predicts Performance of Convolutional Neural Networks](https://mlsanity.com/paper/2206.13100/) (2022)
- [Graph Flow Matching: Enhancing Image Generation with Neighbor-Aware Flow Fields](https://mlsanity.com/paper/2505.24434/) (2025)
- [Mathematical Modeling and Convergence Analysis of Deep Neural Networks with Dense Layer Connectivities in Deep Learning](https://mlsanity.com/paper/2510.02049/) (2025)
- [Deep Neural Networks and PIDE discretizations](https://mlsanity.com/paper/2108.02430/) (2021)
- [Unsupervised Image Semantic Segmentation through Superpixels and Graph Neural Networks](https://mlsanity.com/paper/2210.11810/) (2022)
- [Stability of implicit neural networks for long-term forecasting in dynamical systems](https://mlsanity.com/paper/2305.17155/) (2023)
- [Control and optimization for Neural Partial Differential Equations in Supervised Learning](https://mlsanity.com/paper/2506.20764/) (2025)
- [Deep-layer limit and stability analysis of the basic forward-backward-splitting induced network (II): learning problems](https://mlsanity.com/paper/2605.27133/) (2026)

## Abstract (arXiv)

Deep convolutional neural networks have revolutionized many machine learning and computer vision tasks, however, some remaining key challenges limit their wider use. These challenges include improving the network's robustness to perturbations of the input image and the limited ``field of view'' of convolution operators. We introduce the IMEXnet that addresses these challenges by adapting semi-implicit methods for partial differential equations. Compared to similar explicit networks, such as residual networks, our network is more stable, which has recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The addition of an implicit step connects all pixels in each channel of the image and therefore addresses the field of view problem while still being comparable to standard convolutions in terms of the number of parameters and computational complexity. We also present a new dataset for semantic segmentation and demonstrate the effectiveness of our architecture using the NYU Depth dataset.