# Causal Deep Learning

arXiv: https://arxiv.org/abs/2301.00314 · HTML: https://mlsanity.com/paper/2301.00314/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 127 references that are in the map.
- First posted on arXiv 1 Jan 2023.

## Builds on (most cited first, 25 of 25)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications](https://mlsanity.com/paper/1511.06530/) (2015)
- [Tensor Ring Decomposition](https://mlsanity.com/paper/1606.05535/) (2016)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [Disentangling factors of variation in deep representations using adversarial training](https://mlsanity.com/paper/1611.03383/) (2016)
- [External Validity: From Do-Calculus to Transportability Across Populations](https://mlsanity.com/paper/1503.01603/) (2015)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Incremental multi-domain learning with network latent tensor factorization](https://mlsanity.com/paper/1904.06345/) (2019)
- [Lower Memory Oblivious (Tensor) Subspace Embeddings with Fewer Random Bits: Modewise Methods for Least Squares](https://mlsanity.com/paper/1912.08294/) (2019)
- [Unconstrained Still/Video-Based Face Verification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1605.02686/) (2016)
- [Compositional Hierarchical Tensor Factorization: Representing Hierarchical Intrinsic and Extrinsic Causal Factors](https://mlsanity.com/paper/1911.04180/) (2019)
- [Efficient Contraction of Large Tensor Networks for Weighted Model Counting through Graph Decompositions](https://mlsanity.com/paper/1908.04381/) (2019)
- [CausalX: Causal Explanations and Block Multilinear Factor Analysis](https://mlsanity.com/paper/2102.12853/) (2021)
- [A Generalized Hierarchical Nonnegative Tensor Decomposition](https://mlsanity.com/paper/2109.14820/) (2021)
- [Compositional Dictionaries for Domain Adaptive Face Recognition](https://mlsanity.com/paper/1308.0271/) (2013)
- [A Fully Tensorized Recurrent Neural Network](https://mlsanity.com/paper/2010.04196/) (2020)
- [A New Approach to Multilinear Dynamical Systems and Control](https://mlsanity.com/paper/2108.13583/) (2021)

## Built on by (most cited first, 2 of 2)

- [Tensor Networks Meet Neural Networks: A Survey and Future Perspectives](https://mlsanity.com/paper/2302.09019/) (2023)
- [LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing](https://mlsanity.com/paper/2505.21732/) (2025)

## Abstract (arXiv)

We derive a set of causal deep neural networks whose architectures are a consequence of tensor (multilinear) factor analysis, a framework that facilitates causal inference. Forward causal questions are addressed with a neural network architecture composed of causal capsules and a tensor transformer. Causal capsules compute a set of invariant causal factor representations, whose interactions are governed by a tensor transformation. Inverse causal questions are addressed with a neural network that implements the multilinear projection algorithm. The architecture reverses the order of operations of a forward neural network and estimates the causes of effects. As an alternative to aggressive bottleneck dimension reduction or regularized regression that may camouflage an inherently underdetermined inverse problem, we prescribe modeling different aspects of the mechanism of data formation with piecewise tensor models whose multilinear projections produce multiple candidate solutions. Our forward and inverse questions may be addressed with shallow architectures, but for computationally scalable solutions, we derive a set of deep neural networks by taking advantage of block algebra. An interleaved kernel hierarchy results in doubly non-linear tensor factor models. The causal neural networks that are a consequence of tensor factor analysis are data agnostic, but are illustrated with facial images. Sequential, parallel and asynchronous parallel computation strategies are described.