mlsanityData to 7 Oct 2026

Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations

arXiv 2112.02290 ↗First posted 4 Dec 2021cs.CV cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Learning visual concepts from raw images without strong supervision is a challenging task. In this work, we show the advantages of prototype representations for understanding and revising the latent space of neural concept learners. For this purpose, we introduce interactive Concept Swapping Networks (iCSNs), a novel framework for learning concept-grounded representations via weak supervision and implicit prototype representations. iCSNs learn to bind conceptual information to specific prototype slots by swapping the latent representations of paired images. This semantically grounded and discrete latent space facilitates human understanding and human-machine interaction. We support this claim by conducting experiments on our novel data set "Elementary Concept Reasoning" (ECR), focusing on visual concepts shared by geometric objects.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 54 of its 86 references that are in the map. First posted on arXiv 4 Dec 2021.

Builds on

54 of 86 references.

References · 54

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Mask R-CNNcited by 7,259
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  7. 2015Fast R-CNNcited by 2,930
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  11. 2016Learning without Forgettingcited by 2,465
  12. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
38 more
  1. 2018Group Normalizationcited by 1,514
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  3. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  6. 2020Object-Centric Learning with Slot Attentioncited by 865
  7. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  8. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  9. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  10. 2018Disentangling by Factorisingcited by 614
  11. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  12. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  13. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  14. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  15. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
  16. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  17. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  18. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  19. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  20. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  21. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  22. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  23. 2020Adversarial Continual Learningcited by 140
  24. 2018Recent Advances in Autoencoder-Based Representation Learningcited by 130
  25. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  26. 2017Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observationscited by 100
  27. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  28. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  29. 2021Unsupervised Learning of Compositional Energy Conceptscited by 71
  30. 2020Measuring Disentanglement: A Review of Metricscited by 69
  31. 2020Multimodal Prototypical Networks for Few-shot Learningcited by 48
  32. 2021Logic Explained Networkscited by 45
  33. 2020Self-Supervised Relational Reasoning for Representation Learningcited by 37
  34. 2021Where and What? Examining Interpretable Disentangled Representationscited by 27
  35. 2021Prototype Guided Federated Learning of Visual Feature Representationscited by 22
  36. 2021Evaluating the Interpretability of Generative Models by Interactive Reconstructioncited by 16
  37. 2020Disentangling 3D Prototypical Networks For Few-Shot Concept Learningcited by 15
  38. 2020Contextual Semantic Interpretabilitycited by 14

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 8; 2024: 6; 2025: 3; 2026: 3.

24 papers.

Sort:

By area:

  1. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  2. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  3. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  4. 2023Learning Bottleneck Concepts in Image Classificationcited by 49
  5. 2022Neural Systematic Bindercited by 37
  6. 2023Learning to Intervene on Concept Bottleneckscited by 29
  7. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  8. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  9. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  10. 2024Neural Concept Bindercited by 14
  11. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  12. 2023NAISR: A 3D Neural Additive Model for Interpretable Shape Representationcited by 7
12 more
  1. 2024The KANDY Benchmark: Incremental Neuro-Symbolic Learning and Reasoning with Kandinsky Patternscited by 5
  2. 2022Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Modelscited by 4
  3. 2023Compositionally Equivariant Representation Learningcited by 2
  4. 2023Learning Concept-Based Causal Transition and Symbolic Reasoning for Visual Planningcited by 2
  5. 2025Personalized Interpretability -- Interactive Alignment of Prototypical Parts Networkscited by 2
  6. 2025Debugging Concept Bottleneck Models through Removal and Retrainingcited by 2
  7. 2024ConceptHash: Interpretable Fine-Grained Hashing via Concept Discoverycited by 1
  8. 2023Improving SCGAN's Similarity Constraint and Learning a Better Disentangled Representationcited by 0
  9. 2025Concept Learning for Cooperative Multi-Agent Reinforcement Learningcited by 0
  10. 2026GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learningcited by 0
  11. 2026Learning to Assemble Novel Structures with Unfamiliar Parts under Semantic Constraintscited by 0
  12. 2026Escaping Local Views: Discovering Latent Concepts for Interpretable Multi-Agent Reinforcement Learningcited by 0