mlsanityData to 7 Oct 2026

How to represent part-whole hierarchies in a neural network

arXiv 2102.12627 ↗First posted 25 Feb 2021cs.CVcited by 123

Not on the map. Why?

Abstract (arXiv)

This paper does not describe a working system. Instead, it presents a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM. The advances include transformers, neural fields, contrastive representation learning, distillation and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy which has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 26 of its 67 references that are in the map. First posted on arXiv 25 Feb 2021.

Builds on

26 of 67 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  11. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  12. 2020Object-Centric Learning with Slot Attentioncited by 865
14 more
  1. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  2. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  3. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  4. 2017Dynamic Routing Between Capsulescited by 689
  5. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  6. 2016Using Fast Weights to Attend to the Recent Pastcited by 161
  7. 2019NASA: Neural Articulated Shape Approximationcited by 149
  8. 2019Stacked Capsule Autoencoderscited by 82
  9. 2014Grammar as a Foreign Languagecited by 70
  10. 2020Learning Physical Graph Representations from Visual Scenescited by 50
  11. 2020ISD: Self-Supervised Learning by Iterative Similarity Distillationcited by 33
  12. 2020Canonical Capsules: Self-Supervised Capsules in Canonical Posecited by 30
  13. 2020Unsupervised part representation by Flow Capsulescited by 25
  14. 2019Geometric Capsule Autoencoders for 3D Point Cloudscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 31; 2023: 28; 2024: 21; 2025: 6; 2026: 7.

123 papers.

Sort:

By area:

  1. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  2. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  3. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  4. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  5. 2022Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNscited by 155
  6. 2021Illiterate DALL-E Learns to Composecited by 147
  7. 2022SemMAE: Semantic-Guided Masking for Learning Masked Autoencoderscited by 113
  8. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  9. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 106
  10. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
  11. 2023White-Box Transformers via Sparse Rate Reductioncited by 77
  12. 2021Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domaincited by 70
38 more
  1. 2021TermiNeRF: Ray Termination Prediction for Efficient Neural Renderingcited by 70
  2. 2021A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challengescited by 68
  3. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  4. 2022Hyperbolic Contrastive Learning for Visual Representations beyond Objectscited by 58
  5. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  6. 2021Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fieldscited by 48
  7. 2023Provably Learning Object-Centric Representationscited by 45
  8. 2023Neither hype nor gloom do DNNs justicecited by 40
  9. 2023Understanding Masked Autoencoders via Hierarchical Latent Variable Modelscited by 39
  10. 2022Point Primitive Transformer for Long-Term 4D Point Cloud Video Understandingcited by 35
  11. 2021PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoningcited by 34
  12. 2022Reversible Column Networkscited by 32
  13. 2023HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentationcited by 31
  14. 2021PartGlot: Learning Shape Part Segmentation from Language Reference Gamescited by 30
  15. 2023Rotating Features for Object Discoverycited by 29
  16. 2022Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learningcited by 27
  17. 2021GraphFPN: Graph Feature Pyramid Network for Object Detectioncited by 25
  18. 2023Emergence of Segmentation with Minimalistic White-Box Transformerscited by 23
  19. 2021VLGrammar: Grounded Grammar Induction of Vision and Languagecited by 22
  20. 2021Object-Aware Cropping for Self-Supervised Learningcited by 22
  21. 2024Reversible Decoupling Network for Single Image Reflection Removalcited by 22
  22. 2021Visual Parser: Representing Part-whole Hierarchies with Transformerscited by 21
  23. 2023Cross-Layer Retrospective Retrieving via Layer Attentioncited by 20
  24. 2023Modeling Entities as Semantic Points for Visual Information Extraction in the Wildcited by 20
  25. 2023Grounded Object Centric Learningcited by 17
  26. 2022Self-supervision through Random Segments with Autoregressive Coding (RandSAC)cited by 16
  27. 2023FIT: Far-reaching Interleaved Transformerscited by 16
  28. 2022Unsupervised Segmentation in Real-World Images via Spelke Object Inferencecited by 15
  29. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  30. 2021Persistent Message Passingcited by 13
  31. 2023Unsupervised Open-Vocabulary Object Localization in Videoscited by 13
  32. 2024Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervisioncited by 12
  33. 2022Finding Biological Plausibility for Adversarially Robust Features via Metameric Taskscited by 11
  34. 2022Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networkscited by 11
  35. 2022Towards Self-Supervised Learning of Global and Object-Centric Representationscited by 11
  36. 2022Learning with Capsules: A Surveycited by 10
  37. 2022Active Predictive Coding: A Unified Neural Framework for Learning Hierarchical World Models for Perception and Planningcited by 10
  38. 2024Wavelet-Based Image Tokenizer for Vision Transformerscited by 10

50 most cited of 123 All 123 (JSON)