# Causal Contrastive Learning for Counterfactual Regression Over Time

arXiv: https://arxiv.org/abs/2406.00535 · HTML: https://mlsanity.com/paper/2406.00535/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 84 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 30 of 52)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information](https://mlsanity.com/paper/2006.12013/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Simultaneous Deep Transfer Across Domains and Tasks](https://mlsanity.com/paper/1510.02192/) (2015)
- [Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations](https://mlsanity.com/paper/2002.04083/) (2020)
- [Contrastive learning, multi-view redundancy, and linear models](https://mlsanity.com/paper/2008.10150/) (2020)
- [Causal Transformer for Estimating Counterfactual Outcomes](https://mlsanity.com/paper/2204.07258/) (2022)
- [What makes instance discrimination good for transfer learning?](https://mlsanity.com/paper/2006.06606/) (2020)
- [Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models](https://mlsanity.com/paper/2011.01625/) (2020)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Understanding Contrastive Learning Requires Incorporating Inductive Biases](https://mlsanity.com/paper/2202.14037/) (2022)
- [Tighter Variational Bounds are Not Necessarily Better](https://mlsanity.com/paper/1802.04537/) (2018)
- [Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders](https://mlsanity.com/paper/1902.00450/) (2019)
- [Factorized Contrastive Learning: Going Beyond Multi-view Redundancy](https://mlsanity.com/paper/2306.05268/) (2023)
- [The Unusual Effectiveness of Averaging in GAN Training](https://mlsanity.com/paper/1806.04498/) (2018)

## Built on by (most cited first, 7 of 7)

- [Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation](https://mlsanity.com/paper/2506.21154/) (2025)
- [Beyond Patient Invariance: Learning Cardiac Dynamics via Action-Conditioned JEPAs](https://mlsanity.com/paper/2604.22618/) (2026)
- [CAETC: Causal Autoencoding and Treatment Conditioning for Counterfactual Estimation over Time](https://mlsanity.com/paper/2603.11565/) (2026)
- [Contrastive Representation Learning Helps Cross-institutional Knowledge Transfer: A Study in Pediatric Ventilation Management](https://mlsanity.com/paper/2501.13587/) (2025)
- [Inferring Effects of Major Events through Discontinuity Forecasting of Population Anxiety](https://mlsanity.com/paper/2508.21722/) (2025)
- [From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction](https://mlsanity.com/paper/2605.16927/) (2026)
- [Causal State-Space Model for Causal Inference: Estimating Longitudinal Individual Treatment Effects](https://mlsanity.com/paper/2608.08288/) (2026)

## Abstract (arXiv)

Estimating treatment effects over time holds significance in various domains, including precision medicine, epidemiology, economy, and marketing. This paper introduces a unique approach to counterfactual regression over time, emphasizing long-term predictions. Distinguishing itself from existing models like Causal Transformer, our approach highlights the efficacy of employing RNNs for long-term forecasting, complemented by Contrastive Predictive Coding (CPC) and Information Maximization (InfoMax). Emphasizing efficiency, we avoid the need for computationally expensive transformers. Leveraging CPC, our method captures long-term dependencies in the presence of time-varying confounders. Notably, recent models have disregarded the importance of invertible representation, compromising identification assumptions. To remedy this, we employ the InfoMax principle, maximizing a lower bound of mutual information between sequence data and its representation. Our method achieves state-of-the-art counterfactual estimation results using both synthetic and real-world data, marking the pioneering incorporation of Contrastive Predictive Encoding in causal inference.