mlsanityData to 7 Oct 2026

Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA

arXiv 1605.06336 ↗First posted 20 May 2016stat.ML cs.LGcited by 270

Not on the map. Why?

Abstract (arXiv)

Nonlinear independent component analysis (ICA) provides an appealing framework for unsupervised feature learning, but the models proposed so far are not identifiable. Here, we first propose a new intuitive principle of unsupervised deep learning from time series which uses the nonstationary structure of the data. Our learning principle, time-contrastive learning (TCL), finds a representation which allows optimal discrimination of time segments (windows). Surprisingly, we show how TCL can be related to a nonlinear ICA model, when ICA is redefined to include temporal nonstationarities. In particular, we show that TCL combined with linear ICA estimates the nonlinear ICA model up to point-wise transformations of the sources, and this solution is unique --- thus providing the first identifiability result for nonlinear ICA which is rigorous, constructive, as well as very general.

Key facts. Cited by 270 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 41; 2023: 57; 2024: 41; 2025: 48; 2026: 49.

270 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  2. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  3. 2023A Cookbook of Self-Supervised Learningcited by 248
  4. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 223
  5. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  6. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  7. 2022Interventional Causal Representation Learningcited by 140
  8. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 132
  9. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  10. 2021Independent mechanism analysis, a new concept?cited by 96
  11. 2021Learning Temporally Causal Latent Processes from General Temporal Datacited by 96
  12. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
38 more
  1. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  2. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 84
  3. 2021Unsupervised Learning of Compositional Energy Conceptscited by 71
  4. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  5. 2023Deep Learning for Time Series Classification and Extrinsic Regression: A Current Surveycited by 68
  6. 2024Causal Representation Learning from Multiple Distributions: A General Settingcited by 67
  7. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  8. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 64
  9. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  10. 2022Temporally Disentangled Representation Learningcited by 59
  11. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  12. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
  13. 2023Identifiability Results for Multimodal Contrastive Learningcited by 58
  14. 2023Causal Component Analysiscited by 56
  15. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  16. 2022Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Seriescited by 54
  17. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  18. 2023Time Series Contrastive Learning with Information-Aware Augmentationscited by 52
  19. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  20. 2022Causal Representation Learning for Instantaneous and Temporal Effects in Interactive Systemscited by 50
  21. 2022Mechanistic Mode Connectivitycited by 50
  22. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  23. 2022Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformercited by 47
  24. 2023Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignmentcited by 46
  25. 2023Provably Learning Object-Centric Representationscited by 45
  26. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  27. 2023Causal Discovery from Temporal Data: An Overview and New Perspectivescited by 43
  28. 2021Nonlinear Invariant Risk Minimization: A Causal Approachcited by 42
  29. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  30. 2024InfoNCE: Identifying the Gap Between Theory and Practicecited by 39
  31. 2023BISCUIT: Causal Representation Learning from Binary Interactionscited by 38
  32. 2021Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspectivecited by 37
  33. 2022Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matricescited by 37
  34. 2023Subspace Identification for Multi-Source Domain Adaptationcited by 36
  35. 2021Intra-Inter Subject Self-supervised Learning for Multivariate Cardiac Signalscited by 35
  36. 2021Contrastively Disentangled Sequential Variational Autoencodercited by 34
  37. 2023Learning nonparametric latent causal graphs with unknown interventionscited by 34
  38. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 34

50 most cited of 270 All 270 (JSON)