mlsanityData to 7 Oct 2026

Robustness of Nonlinear Representation Learning

arXiv 2503.15355 ↗First posted 19 Mar 2025stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We study the problem of unsupervised representation learning in slightly misspecified settings, and thus formalize the study of robustness of nonlinear representation learning. We focus on the case where the mixing is close to a local isometry in a suitable distance and show based on existing rigidity results that the mixing can be identified up to linear transformations and small errors. In a second step, we investigate Independent Component Analysis (ICA) with observations generated according to x=f(s)=As+h(s) where A is an invertible mixing matrix and h a small perturbation. We show that we can approximately recover the matrix A and the independent components. Together, these two results show approximate identifiability of nonlinear ICA with almost isometric mixing functions. Those results are a step towards identifiability results for unsupervised representation learning for real-world data that do not follow restrictive model classes.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 55 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

19 of 55 references.

References · 19

  1. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  2. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  3. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  4. 2022Interventional Causal Representation Learningcited by 140
  5. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  6. 2021Independent mechanism analysis, a new concept?cited by 96
  7. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  8. 2018Variational Autoencoders Pursue PCA Directions (by Accident)cited by 84
  9. 2022Linear Causal Disentanglement via Interventionscited by 84
  10. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  11. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
  12. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
7 more
  1. 2023Score-based Causal Representation Learning with Interventionscited by 49
  2. 2020Orthogonal Over-Parameterized Trainingcited by 44
  3. 2020Deep Isometric Learning for Visual Recognitioncited by 38
  4. 2022Disentanglement of Correlated Factors via Hausdorff Factorized Supportcited by 37
  5. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  6. 2022Embrace the Gap: VAEs Perform Independent Mechanism Analysiscited by 28
  7. 2014A convergence and asymptotic analysis of the generalized symmetric FastICA algorithmcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2026When Does LeJEPA Learn a World Model?cited by 27
  2. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  3. 2025When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspectivecited by 4
  4. 2026Statistical and structural identifiability in representation learningcited by 4
  5. 2025Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimalitycited by 0
  6. 2026Logit Distance Bounds Representational Similaritycited by 0
  7. 2026Beyond identifiability: Learning causal representations with few environments and finite samplescited by 0