mlsanityData to 7 Oct 2026

On Contrastive Representations of Stochastic Processes

arXiv 2106.10052 ↗First posted 18 Jun 2021stat.ML cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Learning representations of stochastic processes is an emerging problem in machine learning with applications from meta-learning to physical object models to time series. Typical methods rely on exact reconstruction of observations, but this approach breaks down as observations become high-dimensional or noise distributions become complex. To address this, we propose a unifying framework for learning contrastive representations of stochastic processes (CReSP) that does away with exact reconstruction. We dissect potential use cases for stochastic process representations, and propose methods that accommodate each. Empirically, we show that our methods are effective for learning representations of periodic functions, 3D objects and dynamical processes. Our methods tolerate noisy high-dimensional observations better than traditional approaches, and the learned representations transfer to a range of downstream tasks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 18 Jun 2021.

Builds on

28 of 53 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2016Layer Normalizationcited by 5,579
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2019Contrastive Multiview Codingcited by 1,297
  12. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
16 more
  1. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  2. 2018Image Transformercited by 770
  3. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  4. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  5. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  6. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  7. 2018Conditional Neural Processescited by 421
  8. 2018Neural Processescited by 307
  9. 2019Attentive Neural Processescited by 280
  10. 2019Revisiting Self-Supervised Visual Representation Learningcited by 250
  11. 2020On Contrastive Learning for Likelihood-free Inferencecited by 96
  12. 2017Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributionscited by 24
  13. 2020Function Contrastive Learning of Transferable Meta-Representationscited by 19
  14. 2020A Simple Framework for Uncertainty in Contrastive Learningcited by 16
  15. 2020Improving Transformation Invariance in Contrastive Representation Learningcited by 16
  16. 2019Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddingscited by 10

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 5; 2024: 3.

12 papers.

  1. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 126
  2. 2022Latent Bottlenecked Attentive Neural Processescited by 26
  3. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  4. 2022Contrastive Conditional Neural Processescited by 11
  5. 2023Deep Stochastic Processes via Functional Markov Transition Operatorscited by 7
  6. 2023Modality-Agnostic Self-Supervised Learning with Meta-Learned Masked Auto-Encodercited by 5
  7. 2024Spectral Convolutional Conditional Neural Processescited by 4
  8. 2023Adaptive Conditional Quantile Neural Processescited by 3
  9. 2024Learning to Learn with Contrastive Meta-Objectivecited by 3
  10. 2023Adversarially Contrastive Estimation of Conditional Neural Processescited by 1
  11. 2023Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processescited by 0
  12. 2024BBScoreV2: Learning Time-Evolution and Latent Alignment from Stochastic Representationcited by 0