mlsanityData to 7 Oct 2026

Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck

arXiv 2309.07200 ↗First posted 13 Sep 2023cs.LG cs.AI cs.ITcited by 6

Not on the map. Why?

Abstract (arXiv)

Markov processes are widely used mathematical models for describing dynamic systems in various fields. However, accurately simulating large-scale systems at long time scales is computationally expensive due to the short time steps required for accurate integration. In this paper, we introduce an inference process that maps complex systems into a simplified representational space and models large jumps in time. To achieve this, we propose Time-lagged Information Bottleneck (T-IB), a principled objective rooted in information theory, which aims to capture relevant temporal features while discarding high-frequency information to simplify the simulation task and minimize the inference error. Our experiments demonstrate that T-IB learns information-optimal representations for accurately modeling the statistical properties and dynamics of the original process at a selected time lag, outperforming existing time-lagged dimensionality reduction methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 37 of its 78 references that are in the map. First posted on arXiv 13 Sep 2023.

Builds on

37 of 78 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2018Group Normalizationcited by 1,514
  5. 2016Variational Graph Auto-Encoderscited by 1,373
  6. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  7. 2018Deep Graph Infomaxcited by 1,101
  8. 2016Deep Variational Information Bottleneckcited by 915
  9. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  10. 2019On Variational Bounds of Mutual Informationcited by 550
  11. 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
  12. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
25 more
  1. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  2. 2020Learning Robust Representations via Multi-View Information Bottleneckcited by 189
  3. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
  4. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  5. 2022Earthformer: Exploring Space-Time Transformers for Earth System Forecastingcited by 146
  6. 2019Understanding the Limitations of Variational Mutual Information Estimatorscited by 140
  7. 2017VAMPnets: Deep learning of molecular kineticscited by 135
  8. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
  9. 2020The Conditional Entropy Bottleneckcited by 75
  10. 2023Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamicscited by 72
  11. 2018Machine Learning of coarse-grained Molecular Dynamics Force Fieldscited by 68
  12. 2017Time-lagged autoencoders: Deep learning of slow collective variables for molecular kineticscited by 64
  13. 2023Timewarp: Transferable Acceleration of Molecular Dynamics by Learning Time-Coarsened Dynamicscited by 62
  14. 2017Variational approach for learning Markov processes from time series datacited by 51
  15. 2021Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspectivecited by 37
  16. 2021Deeptime: a Python library for machine learning dynamical models from time series datacited by 36
  17. 2022Flow-matching -- efficient coarse-graining of molecular dynamics without forcescited by 36
  18. 2017Variational Encoding of Complex Dynamicscited by 21
  19. 2019Spatio-Temporal Deep Graph Infomaxcited by 21
  20. 2018Deep Generative Markov State Modelscited by 19
  21. 2022TempCLR: Temporal Alignment Representation with Contrastive Learningcited by 15
  22. 2023Implicit Transfer Operator Learning: Multiple Time-Resolution Surrogates for Molecular Dynamicscited by 14
  23. 2020Molecular Latent Space Simulatorscited by 13
  24. 2023On the Effectiveness of Hybrid Mutual Information Estimationcited by 6
  25. 2019Capabilities and Limitations of Time-lagged Autoencoders for Slow Mode Discovery in Dynamical Systemscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systemscited by 7
  2. 2025Information Shapes Koopman Representationcited by 3
  3. 2025Predicting the Energy Landscape of Stochastic Dynamical System via Physics-informed Self-supervised Learningcited by 0
  4. 2025InfoDPCCA: Information-Theoretic Dynamic Probabilistic Canonical Correlation Analysiscited by 0
  5. 2025Tracking Temporal Dynamics of Vector Sets with Gaussian Processcited by 0
  6. 2026Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systemscited by 0