What Converges in the Platonic Representation Hypothesis? Structure over Geometry
Not on the map. Why?
Abstract (arXiv)
The Platonic Representation Hypothesis suggests that increasingly capable models converge toward shared representations. Recent work narrows this claim to shared local neighborhood relationships, finding that capacity-dependent trends in several global similarity measures largely disappear after calibration. We challenge this interpretation by showing that prior local-global comparisons confound structural scale (local versus global) with what is compared: relational structure, defined by which samples are related, versus metric geometry, characterized by quantitative relations such as distances, similarities, or correlations. To disentangle these factors, we construct a controlled 2×2 framework that evaluates both relational structure and metric geometry at local and global scales. We introduce H0 skeleton overlap as a global counterpart to mutual k-nearest neighbors, together with matched distance-aware variants. Across vision-language models, relational structure exhibits robust representational convergence at both scales after calibration, whereas increasingly stringent distance agreement substantially weakens alignment and progressively flattens the capacity-dependent trend. We further extend the analysis beyond ambient Euclidean geometry by evaluating distance agreement under a Riemannian metric approximation and recover the same structure-geometry pattern. The pattern is also reproduced in video-text representations. Together, these results show that relational convergence extends beyond local neighborhoods to global spanning structure, whereas metric geometry exhibits substantially weaker convergence.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
28 of 44 references.
References · 28
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
- 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
- 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
- 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
16 more
- 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
- 2015Convergent Learning: Do different neural networks learn the same representations?cited by 220
- 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
- 2023Getting aligned on representational alignmentcited by 143
- 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
- 2019Topological Autoencoderscited by 127
- 2021Generalized Shape Metrics on Neural Representationscited by 112
- 2025PerceptionLM: Open-Access Data and Models for Detailed Visual Understandingcited by 92
- 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
- 2018Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representationcited by 62
- 2026Revisiting the Platonic Representation Hypothesis: An Aristotelian Viewcited by 44
- 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 31
- 2025RTD-Lite: Scalable Topological Analysis for Comparing Weighted Graphs in Learning Taskscited by 4
- 2025Topological Alignment of Shared Vision-Language Embedding Spacecited by 2
- 2026Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learningcited by 1
- 2026Scalable and Interpretable Representation Alignment with Ordinal Similaritycited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Platonic Representation Hypothesis on World Models
- 2025Unraveling the geometry of visual relational reasoning
- 2026Representation Alignment Rests on Linear Structure
- 2025Escaping Plato's Cave: JAM for Aligning Independently Trained Vision and Language Models
- 2026Barycentric alignment for instance-level comparison of neural representations
- 2026Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds
- 2026Transformer Geometry Observatory TGO-III: Semantic Geometry Observatory
- 2025Do Large Language Models Truly Understand Geometric Structures?