mlsanityData to 14 Oct 2026

Identifiability of a statistical model with two latent vectors: Importance of the dimensionality relation and application to graph embedding

arXiv 2405.19760 ↗First posted 30 May 2024stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Identifiability of statistical models is a key notion in unsupervised representation learning. Recent work of nonlinear independent component analysis (ICA) employs auxiliary data and has established identifiable conditions. This paper proposes a statistical model of two latent vectors with single auxiliary data generalizing nonlinear ICA, and establishes various identifiability conditions. Unlike previous work, the two latent vectors in the proposed model can have arbitrary dimensions, and this property enables us to reveal an insightful dimensionality relation among two latent vectors and auxiliary data in identifiability conditions. Furthermore, surprisingly, we prove that the indeterminacies of the proposed model has the same as linear ICA under certain conditions: The elements in the latent vector can be recovered up to their permutation and scales. Next, we apply the identifiability theory to a statistical model for graph data. As a result, one of the identifiability conditions includes an appealing implication: Identifiability of the statistical model could depend on the maximum value of link weights in graph data. Then, we propose a practical method for identifiable graph embedding. Finally, we numerically demonstrate that the proposed method well-recovers the latent vectors and model identifiability clearly depends on the maximum value of link weights, which supports the implication of our theoretical results

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 48 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

30 of 48 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2018Deep contextualized word representationscited by 2,290
  4. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  5. 2018MINE: Mutual Information Neural Estimationcited by 683
  6. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  7. 2017Look, Listen and Learncited by 458
  8. 2022Multimodal Learning with Transformers: A Surveycited by 400
  9. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  10. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  11. 2017A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applicationscited by 260
  12. 2020Weakly-Supervised Disentanglement Without Compromisescited by 239
18 more
  1. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 122
  2. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  3. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  4. 2021Independent mechanism analysis, a new concept?cited by 97
  5. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  6. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 85
  7. 2016Normalizing Flows on Riemannian Manifoldscited by 68
  8. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  9. 2019Causal Discovery with General Non-Linear Relationships Using Non-Linear ICAcited by 58
  10. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  11. 2020Few-shot Domain Adaptation by Causal Mechanism Transfercited by 47
  12. 2021Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspectivecited by 37
  13. 2023Generalizing Nonlinear ICA Beyond Structural Sparsitycited by 34
  14. 2020Causal Mosaic: Cause-Effect Inference via Nonlinear ICA and Ensemble Methodcited by 16
  15. 2018A probabilistic framework for multi-view feature learning with many-to-many associations via neural networkscited by 5
  16. 2021Representation learning for maximization of MI, nonlinear ICA and nonlinear subspaces with robust density ratio estimationcited by 4
  17. 2018Neural-Kernelized Conditional Density Estimationcited by 3
  18. 2019Robust Graph Embedding with Noisy Link Weightscited by 3

Built on by

Not cited yet.