Nonparametric Identification of Latent Concepts
Not on the map. Why?
Abstract (arXiv)
We are born with the ability to learn concepts by comparing diverse observations. This helps us to understand the new world in a compositional manner and facilitates extrapolation, as objects naturally consist of multiple concepts. In this work, we argue that the cognitive mechanism of comparison, fundamental to human learning, is also vital for machines to recover true concepts underlying the data. This offers correctness guarantees for the field of concept learning, which, despite its impressive empirical successes, still lacks general theoretical support. Specifically, we aim to develop a theoretical framework for the identifiability of concepts with multiple classes of observations. We show that with sufficient diversity across classes, hidden concepts can be identified without assuming specific concept types, functional relations, or parametric generative models. Interestingly, even when conditions are not globally satisfied, we can still provide alternative guarantees for as many concepts as possible based on local comparisons, thereby extending the applicability of our theory to more flexible scenarios. Moreover, the hidden structure between classes and concepts can also be identified nonparametrically. We validate our theoretical results in both synthetic and real-world settings.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 48 of its 79 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
48 of 79 references.
References · 48
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
- 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
- 2020Concept Bottleneck Modelscited by 1,022
- 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
- 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
36 more
- 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
- 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
- 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
- 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
- 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
- 2023Causal-learn: Causal Discovery in Pythoncited by 108
- 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
- 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
- 2020Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)cited by 91
- 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 84
- 2024On the Origins of Linear Representations in Large Language Modelscited by 76
- 2024Causal Representation Learning from Multiple Distributions: A General Settingcited by 67
- 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
- 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
- 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
- 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
- 2023Provably Learning Object-Centric Representationscited by 45
- 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
- 2024Compositional Generative Modeling: A Single Model is Not All You Needcited by 41
- 2023Subspace Identification for Multi-Source Domain Adaptationcited by 36
- 2023Generalizing Nonlinear ICA Beyond Structural Sparsitycited by 33
- 2020Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representationscited by 32
- 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 30
- 2020Unsupervised Discovery of 3D Physical Objects from Videocited by 27
- 2021The Role of Explainability in Assuring Safety of Machine Learning in Healthcarecited by 26
- 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
- 2023Object-centric architectures enable efficient causal representation learningcited by 21
- 2022Concept Gradient: Concept-based Interpretation Without Linear Assumptioncited by 20
- 2023Concept Learning for Interpretable Multi-Agent Reinforcement Learningcited by 18
- 2021Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularizationcited by 17
- 2022When are Post-hoc Conceptual Explanations Identifiable?cited by 14
- 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
- 2022Provable concept learning for interpretable predictions using variational autoencoderscited by 9
- 2024Detecting and Identifying Selection Structure in Sequential Datacited by 9
- 2023Generalized Precision Matrix for Scalable Estimation of Nonparametric Markov Networkscited by 7
- 2025Nonparametric Factor Analysis and Beyondcited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.