# Invariant multiscale neural networks for data-scarce scientific applications

arXiv: https://arxiv.org/abs/2406.08318 · HTML: https://mlsanity.com/paper/2406.08318/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 12 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Convolutional Neural Operators for robust and accurate learning of PDEs](https://mlsanity.com/paper/2302.01178/) (2023)
- [Understanding Convolution for Semantic Segmentation](https://mlsanity.com/paper/1702.08502/) (2017)
- [Generalization capabilities of translationally equivariant neural networks](https://mlsanity.com/paper/2103.14686/) (2021)
- [Deep Reinforcement Learning Optimizes Graphene Nanopores for Efficient Desalination](https://mlsanity.com/paper/2101.07399/) (2021)
- [Surrogate- and invariance-boosted contrastive learning for data-scarce applications in science](https://mlsanity.com/paper/2110.08406/) (2021)
- [Computing excited states of molecules using normalizing flows](https://mlsanity.com/paper/2308.16468/) (2023)
- [Deep learning extraction of band structure parameters from density of states: a case study on trilayer graphene](https://mlsanity.com/paper/2210.06310/) (2022)
- [Towards Neural Variational Monte Carlo That Scales Linearly with System Size](https://mlsanity.com/paper/2212.11296/) (2022)
- [Implementing scalable matrix-vector products for the exact diagonalization methods in quantum many-body physics](https://mlsanity.com/paper/2308.16712/) (2023)

## Abstract (arXiv)

Success of machine learning (ML) in the modern world is largely determined by abundance of data. However at many industrial and scientific problems, amount of data is limited. Application of ML methods to data-scarce scientific problems can be made more effective via several routes, one of them is equivariant neural networks possessing knowledge of symmetries. Here we suggest that combination of symmetry-aware invariant architectures and stacks of dilated convolutions is a very effective and easy to implement receipt allowing sizable improvements in accuracy over standard approaches. We apply it to representative physical problems from different realms: prediction of bandgaps of photonic crystals, and network approximations of magnetic ground states. The suggested invariant multiscale architectures increase expressibility of networks, which allow them to perform better in all considered cases.