# Neural Information Squeezer for Causal Emergence

arXiv: https://arxiv.org/abs/2201.10154 · HTML: https://mlsanity.com/paper/2201.10154/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 21 of 21)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)
- [Causal Consistency of Structural Equation Models](https://mlsanity.com/paper/1707.00819/) (2017)
- [Discovering physical concepts with neural networks](https://mlsanity.com/paper/1807.10300/) (2018)
- [Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data](https://mlsanity.com/paper/2006.10833/) (2020)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [Approximate Causal Abstraction](https://mlsanity.com/paper/1906.11583/) (2019)
- [Neural Network Renormalization Group](https://mlsanity.com/paper/1802.02840/) (2018)
- [Mutual Information, Neural Networks and the Renormalization Group](https://mlsanity.com/paper/1704.06279/) (2017)
- [Universal Approximation Property of Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.02414/) (2020)
- [When the map is better than the territory](https://mlsanity.com/paper/1612.09592/) (2016)
- [A General Deep Learning Framework for Network Reconstruction and Dynamics Learning](https://mlsanity.com/paper/1812.11482/) (2018)
- [Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data](https://mlsanity.com/paper/2004.08220/) (2020)
- [Discovering State Variables Hidden in Experimental Data](https://mlsanity.com/paper/2112.10755/) (2021)
- [RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior](https://mlsanity.com/paper/2010.00029/) (2020)
- [The emergence of informative higher scales in complex networks](https://mlsanity.com/paper/1907.03902/) (2019)

## Built on by (most cited first, 3 of 3)

- [Finding emergence in data by maximizing effective information](https://mlsanity.com/paper/2308.09952/) (2023)
- [Data driven modeling for self-similar dynamics](https://mlsanity.com/paper/2310.08282/) (2023)
- [Partial Effective Information Decomposition for Synergistic Causality](https://mlsanity.com/paper/2605.03267/) (2026)

## Abstract (arXiv)

The classic studies of causal emergence have revealed that in some Markovian dynamical systems, far stronger causal connections can be found on the higher-level descriptions than the lower-level of the same systems if we coarse-grain the system states in an appropriate way. However, identifying this emergent causality from the data is still a hard problem that has not been solved because the correct coarse-graining strategy can not be found easily. This paper proposes a general machine learning framework called Neural Information Squeezer to automatically extract the effective coarse-graining strategy and the macro-state dynamics, as well as identify causal emergence directly from the time series data. By decomposing a coarse-graining operation into two processes: information conversion and information dropping out, we can not only exactly control the width of the information channel, but also can derive some important properties analytically including the exact expression of the effective information of a macro-dynamics. We also show how our framework can extract the dynamics on different levels and identify causal emergence from the data on several exampled systems.