# InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysis

arXiv: https://arxiv.org/abs/2506.08884 · HTML: https://mlsanity.com/paper/2506.08884/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [The Conditional Entropy Bottleneck](https://mlsanity.com/paper/2002.05379/) (2020)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [Deep Variational Canonical Correlation Analysis](https://mlsanity.com/paper/1610.03454/) (2016)
- [Time Series Cluster Kernel for Learning Similarities between Multivariate Time Series with Missing Data](https://mlsanity.com/paper/1704.00794/) (2017)
- [Brain Network Construction and Classification Toolbox (BrainNetClass)](https://mlsanity.com/paper/1906.09908/) (2019)
- [Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck](https://mlsanity.com/paper/2309.07200/) (2023)
- [Non-linear Canonical Correlation Analysis: A Compressed Representation Approach](https://mlsanity.com/paper/1810.13259/) (2018)
- [Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer](https://mlsanity.com/paper/2211.16607/) (2022)
- [Compressed Predictive Information Coding](https://mlsanity.com/paper/2203.02051/) (2022)
- [Which bits went where? Past and future transfer entropy decomposition with the information bottleneck](https://mlsanity.com/paper/2411.04992/) (2024)
- [Deep Dynamic Probabilistic Canonical Correlation Analysis](https://mlsanity.com/paper/2502.05155/) (2025)

## Abstract (arXiv)

Extracting meaningful latent representations from high-dimensional sequential data is a crucial challenge in machine learning, with applications spanning natural science and engineering. We introduce InfoDPCCA, a dynamic probabilistic Canonical Correlation Analysis (CCA) framework designed to model two interdependent sequences of observations. InfoDPCCA leverages a novel information-theoretic objective to extract a shared latent representation that captures the mutual structure between the data streams and balances representation compression and predictive sufficiency while also learning separate latent components that encode information specific to each sequence. Unlike prior dynamic CCA models, such as DPCCA, our approach explicitly enforces the shared latent space to encode only the mutual information between the sequences, improving interpretability and robustness. We further introduce a two-step training scheme to bridge the gap between information-theoretic representation learning and generative modeling, along with a residual connection mechanism to enhance training stability. Through experiments on synthetic and medical fMRI data, we demonstrate that InfoDPCCA excels as a tool for representation learning. Code of InfoDPCCA is available at https://github.com/marcusstang/InfoDPCCA.