# Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow

arXiv: https://arxiv.org/abs/1801.07172 · HTML: https://mlsanity.com/paper/1801.07172/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2018.

## Built on by (most cited first, 15 of 15)

- [Renormalizing Diffusion Models](https://mlsanity.com/paper/2308.12355/) (2023)
- [Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group](https://mlsanity.com/paper/2107.06898/) (2021)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Wavelet Conditional Renormalization Group](https://mlsanity.com/paper/2207.04941/) (2022)
- [Universality of Winning Tickets: A Renormalization Group Perspective](https://mlsanity.com/paper/2110.03210/) (2021)
- [Can a CNN trained on the Ising model detect the phase transition of the q-state Potts model?](https://mlsanity.com/paper/2104.03632/) (2021)
- [Feature extraction of machine learning and phase transition point of Ising model](https://mlsanity.com/paper/2111.11166/) (2021)
- [Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysis](https://mlsanity.com/paper/2410.00396/) (2024)
- [Physics-Guided Problem Decomposition for Scaling Deep Learning of High-dimensional Eigen-Solvers: The Case of Schr"odinger's Equation](https://mlsanity.com/paper/2202.05994/) (2022)
- [Data driven modeling for self-similar dynamics](https://mlsanity.com/paper/2310.08282/) (2023)
- [On the Road to Clarity: Exploring Explainable AI for World Models in a Driver Assistance System](https://mlsanity.com/paper/2404.17350/) (2024)
- [Absolute abstraction: a renormalisation group approach](https://mlsanity.com/paper/2407.01656/) (2024)
- [Symmetry and Generalisation in Neural Approximations of Renormalisation Transformations](https://mlsanity.com/paper/2510.16591/) (2025)
- [Lecture Notes on Statistical Physics and Neural Networks](https://mlsanity.com/paper/2605.06394/) (2026)
- [Renormalization Group Flow Matching for Scalable Local Generative Modeling](https://mlsanity.com/paper/2608.23696/) (2026)

## Abstract (arXiv)

Theoretical understanding of how deep neural network (DNN) extracts features from input images is still unclear, but it is widely believed that the extraction is performed hierarchically through a process of coarse-graining. It reminds us of the basic concept of renormalization group (RG) in statistical physics. In order to explore possible relations between DNN and RG, we use the Restricted Boltzmann machine (RBM) applied to Ising model and construct a flow of model parameters (in particular, temperature) generated by the RBM. We show that the unsupervised RBM trained by spin configurations at various temperatures from T=0 to T=6 generates a flow along which the temperature approaches the critical value Tc=2.27. This behavior is opposite to the typical RG flow of the Ising model. By analyzing various properties of the weight matrices of the trained RBM, we discuss why it flows towards Tc and how the RBM learns to extract features of spin configurations.