# Less Emphasis on Difficult Layer Regions: Curriculum Learning for Singularly Perturbed Convection-Diffusion-Reaction Problems

arXiv: https://arxiv.org/abs/2210.12685 · HTML: https://mlsanity.com/paper/2210.12685/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 23 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images](https://mlsanity.com/paper/1808.01097/) (2018)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Norm-Based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2006.02014/) (2020)
- [Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models](https://mlsanity.com/paper/2108.12956/) (2021)
- [Curriculum Learning of Visual Attribute Clusters for Multi-Task Classification](https://mlsanity.com/paper/1709.06664/) (2017)

## Built on by (most cited first, 3 of 3)

- [More Consistent Accuracy PINN via Alternating Easy-Hard Training](https://mlsanity.com/paper/2512.17607/) (2025)
- [Physics-informed post-processing of stabilized finite element solutions for transient convection-dominated problems](https://mlsanity.com/paper/2603.03259/) (2026)
- [LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems](https://mlsanity.com/paper/2607.03682/) (2026)

## Abstract (arXiv)

Although Physics-Informed Neural Networks (PINNs) have been successfully applied in a wide variety of science and engineering fields, they can fail to accurately predict the underlying solution in slightly challenging convection-diffusion-reaction problems. In this paper, we investigate the reason of this failure from a domain distribution perspective, and identify that learning multi-scale fields simultaneously makes the network unable to advance its training and easily get stuck in poor local minima. We show that the widespread experience of sampling more collocation points in high-loss layer regions hardly help optimize and may even worsen the results. These findings motivate the development of a novel curriculum learning method that encourages neural networks to prioritize learning on easier non-layer regions while downplaying learning on harder layer regions. The proposed method helps PINNs automatically adjust the learning emphasis and thereby facilitate the optimization procedure. Numerical results on typical benchmark equations show that the proposed curriculum learning approach mitigates the failure modes of PINNs and can produce accurate results for very sharp boundary and interior layers. Our work reveals that for equations whose solutions have large scale differences, paying less attention to high-loss regions can be an effective strategy for learning them accurately.