Global Geometry Is Not Enough for Vision Representations
Not on the map. Why?
Abstract (arXiv)
A common assumption in representation learning is that globally well-distributed embeddings support robust and generalizable representations. This focus has shaped both training objectives and evaluation protocols, implicitly treating global geometry as a proxy for representational competence. While global geometry effectively encodes which elements are present, it is often insensitive to how they are composed. We investigate this limitation by testing the ability of geometric metrics to predict compositional binding across a diverse suite of vision encoders. We find that standard geometry-based statistics exhibit near-zero correlation with compositional binding. In contrast, functional sensitivity, as measured by the input--output Jacobian, reliably tracks this capability. We further provide an analytic account showing that this disparity arises from objective design, as existing losses explicitly constrain embedding geometry but leave the local input--output mapping unconstrained. These results suggest that global embedding geometry captures only a partial view of representational competence and establish functional sensitivity as a critical complementary axis for modeling composite structure.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 3 Feb 2026.
Builds on
37 of 50 references.
References · 37
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020Exploring Simple Siamese Representation Learningcited by 2,853
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
25 more
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
- 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
- 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
- 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
- 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
- 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 227
- 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 205
- 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
- 2017Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practicecited by 132
- 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
- 2021Generalized Shape Metrics on Neural Representationscited by 109
- 2022Understanding Collapse in Non-Contrastive Siamese Representation Learningcited by 42
- 2024The Double-Ellipsoid Geometry of CLIPcited by 42
- 2023The SSL Interplay: Augmentations, Inductive Bias, and Generalizationcited by 40
- 2024MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Modelscited by 29
- 2024The Hard Positive Truth about Vision-Language Compositionalitycited by 21
- 2025Is CLIP ideal? No. Can we fix it? Yes!cited by 20
- 2024LDReg: Local Dimensionality Regularized Self-Supervised Learningcited by 11
- 2024Evaluating Representational Similarity Measures from the Lens of Functional Correspondencecited by 11
- 2024Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularizationcited by 9
- 2024Collapse-Proof Non-Contrastive Self-Supervised Learningcited by 7
- 2025Scalable Evaluation and Neural Models for Compositional Generalizationcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.