# Inverse Evolution Layers: Physics-informed Regularizers for Deep Neural Networks

arXiv: https://arxiv.org/abs/2307.07344 · HTML: https://mlsanity.com/paper/2307.07344/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [Adversarial Regularizers in Inverse Problems](https://mlsanity.com/paper/1805.11572/) (2018)
- [Learning deep structured active contours end-to-end](https://mlsanity.com/paper/1803.06329/) (2018)
- [Deep Convolutional Neural Networks with Spatial Regularization, Volume and Star-shape Priori for Image Segmentation](https://mlsanity.com/paper/2002.03989/) (2020)
- [Structure preserving deep learning](https://mlsanity.com/paper/2006.03364/) (2020)
- [DARNet: Deep Active Ray Network for Building Segmentation](https://mlsanity.com/paper/1905.05889/) (2019)
- [On learning optimized reaction diffusion processes for effective image restoration](https://mlsanity.com/paper/1503.05768/) (2015)
- [Convex Shape Prior for Deep Neural Convolution Network based Eye Fundus Images Segmentation](https://mlsanity.com/paper/2005.07476/) (2020)
- [End-to-End Deep Convolutional Active Contours for Image Segmentation](https://mlsanity.com/paper/1909.13359/) (2019)
- [Convex Shape Representation with Binary Labels for Image Segmentation: Models and Fast Algorithms](https://mlsanity.com/paper/2002.09600/) (2020)

## Abstract (arXiv)

Traditional image processing methods employing partial differential equations (PDEs) offer a multitude of meaningful regularizers, along with valuable theoretical foundations for a wide range of image-related tasks. This makes their integration into neural networks a promising avenue. In this paper, we introduce a novel regularization approach inspired by the reverse process of PDE-based evolution models. Specifically, we propose inverse evolution layers (IELs), which serve as bad property amplifiers to penalize neural networks of which outputs have undesired characteristics. Using IELs, one can achieve specific regularization objectives and endow neural networks' outputs with corresponding properties of the PDE models. Our experiments, focusing on semantic segmentation tasks using heat-diffusion IELs, demonstrate their effectiveness in mitigating noisy label effects. Additionally, we develop curve-motion IELs to enforce convex shape regularization in neural network-based segmentation models for preventing the generation of concave outputs. Theoretical analysis confirms the efficacy of IELs as an effective regularization mechanism, particularly in handling training with label issues.