mlsanityData to 7 Oct 2026

Generalizable Information Theoretic Causal Representation

arXiv 2202.08388 ↗First posted 17 Feb 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

It is evidence that representation learning can improve model's performance over multiple downstream tasks in many real-world scenarios, such as image classification and recommender systems. Existing learning approaches rely on establishing the correlation (or its proxy) between features and the downstream task (labels), which typically results in a representation containing cause, effect and spurious correlated variables of the label. Its generalizability may deteriorate because of the unstability of the non-causal parts. In this paper, we propose to learn causal representation from observational data by regularizing the learning procedure with mutual information measures according to our hypothetical causal graph. The optimization involves a counterfactual loss, based on which we deduce a theoretical guarantee that the causality-inspired learning is with reduced sample complexity and better generalization ability. Extensive experiments show that the models trained on causal representations learned by our approach is robust under adversarial attacks and distribution shift.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 54 references that are in the map. First posted on arXiv 17 Feb 2022.

Builds on

23 of 54 references.

References · 23

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2019Invariant Risk Minimizationcited by 1,832
  4. 2016Deep Variational Information Bottleneckcited by 915
  5. 2021Domain Generalization: A Surveycited by 760
  6. 2018MINE: Mutual Information Neural Estimationcited by 680
  7. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  8. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  9. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  10. 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 233
  11. 2018Statistical Aspects of Wasserstein Distancescited by 202
  12. 2017CausalGAN: Learning Causal Implicit Generative Models with Adversarial Trainingcited by 152
11 more
  1. 2018Domain Generalization via Conditional Invariant Representationcited by 146
  2. 2017Heterogeneous Information Network Embedding for Recommendationcited by 133
  3. 2017Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributionscited by 125
  4. 2020CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learningcited by 109
  5. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  6. 2017Learning Independent Causal Mechanismscited by 104
  7. 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
  8. 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
  9. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  10. 2018Generalization in anti-causal learningcited by 23
  11. 2021Top-N Recommendation with Counterfactual User Preference Simulationcited by 21

Built on by

Not cited yet.