mlsanityData to 7 Oct 2026

Representation Topology Divergence: A Method for Comparing Neural Network Representations

arXiv 2201.00058 ↗First posted 31 Dec 2021cs.LG cs.AI math.ATcited by 66

Not on the map. Why?

Abstract (arXiv)

Comparison of data representations is a complex multi-aspect problem that has not enjoyed a complete solution yet. We propose a method for comparing two data representations. We introduce the Representation Topology Divergence (RTD), measuring the dissimilarity in multi-scale topology between two point clouds of equal size with a one-to-one correspondence between points. The data point clouds are allowed to lie in different ambient spaces. The RTD is one of the few TDA-based practical methods applicable to real machine learning datasets. Experiments show that the proposed RTD agrees with the intuitive assessment of data representation similarity and is sensitive to its topological structure. We apply RTD to gain insights on neural networks representations in computer vision and NLP domains for various problems: training dynamics analysis, data distribution shift, transfer learning, ensemble learning, disentanglement assessment.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 31 Dec 2021.

Builds on

27 of 53 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  7. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  8. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  9. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  10. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  11. 2019Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAMLcited by 359
  12. 2018Insights on representational similarity in neural networks with canonical correlationcited by 318
15 more
  1. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  2. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  3. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 215
  4. 2017An introduction to Topological Data Analysis: fundamental and practical aspects for data scientistscited by 171
  5. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  6. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  7. 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 138
  8. 2019Topological Autoencoderscited by 124
  9. 2018Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representationcited by 62
  10. 2020NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processingcited by 60
  11. 2018Geometry Score: A Method For Comparing Generative Adversarial Networkscited by 56
  12. 2020Similarity Analysis of Contextual Word Representation Modelscited by 54
  13. 2019The Shape of Data: Intrinsic Distance for Data Distributionscited by 38
  14. 2020GPU-Accelerated Computation of Vietoris-Rips Persistence Barcodescited by 26
  15. 2020Evaluating the Disentanglement of Deep Generative Models through Manifold Topologycited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 8; 2024: 21; 2025: 15; 2026: 15.

66 papers.

Sort:

By area:

  1. 2022Relative representations enable zero-shot latent space communicationcited by 175
  2. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  3. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  4. 2024Universal Neurons in GPT2 Language Modelscited by 100
  5. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 65
  6. 2023Latent Space Translation via Semantic Alignmentcited by 46
  7. 2024Do Vision and Language Encoders Represent the World Similarly?cited by 44
  8. 2023Learning Topology-Preserving Data Representationscited by 30
  9. 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 28
  10. 2022Wasserstein Iterative Networks for Barycenter Estimationcited by 23
  11. 2022Acceptability Judgements via Examining the Topology of Attention Mapscited by 19
  12. 2024ReSi: A Comprehensive Benchmark for Representational Similarity Measurescited by 18
38 more
  1. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
  2. 2024Objective drives the consistency of representational similarity across datasetscited by 16
  3. 2023Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theorycited by 12
  4. 2023Understanding the Inner Workings of Language Models Through Representation Dissimilaritycited by 12
  5. 2024Persistent Topological Features in Large Language Modelscited by 11
  6. 2022Experimental Observations of the Topology of Convolutional Neural Network Activationscited by 10
  7. 2024Latent Space Translation via Inverse Relative Projectioncited by 10
  8. 2022Topological Data Analysis for Speech Processingcited by 9
  9. 2024Neural Lineagecited by 9
  10. 2023Bootstrapping Parallel Anchors for Relative Representationscited by 7
  11. 2024Mapping the Multiverse of Latent Representationscited by 5
  12. 2025The Flood Complex: Large-Scale Persistent Homology on Millions of Pointscited by 5
  13. 2026Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depthcited by 5
  14. 2023Disentanglement Learning via Topologycited by 4
  15. 2024Deep Regression Representation Learning with Topologycited by 4
  16. 2024Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representationscited by 4
  17. 2025RTD-Lite: Scalable Topological Analysis for Comparing Weighted Graphs in Learning Taskscited by 4
  18. 2025Connecting Neural Models Latent Geometries with Relative Geodesic Representationscited by 4
  19. 2025From Topology to Retrieval: Decoding Embedding Spaces with Unified Signaturescited by 4
  20. 2022Internal Representations of Vision Models Through the Lens of Frames on Data Manifoldscited by 3
  21. 2024Topological Representations of Heterogeneous Learning Dynamics of Recurrent Spiking Neural Networkscited by 3
  22. 2025Topological Autoencoders++: Fast and Accurate Cycle-Aware Dimensionality Reductioncited by 3
  23. 2026Representation Geometry as a Diagnostic for Out-of-Distribution Robustnesscited by 3
  24. 2024Defining Neural Network Architecture through Polytope Structures of Datasetcited by 2
  25. 2024Mixup Barcodes: Quantifying Geometric-Topological Interactions between Point Cloudscited by 2
  26. 2024Diagnosing Neural Convergence with Topological Alignment Spectracited by 2
  27. 2025Manifold Approximation leads to Robust Kernel Alignmentcited by 2
  28. 2023Estimating class separability of text embeddings with persistent homologycited by 1
  29. 2024Beyond Simple Averaging: Improving NLP Ensemble Performance with Topological-Data-Analysis-Based Weightingcited by 1
  30. 2024Latent Functional Maps: a spectral framework for representation alignmentcited by 1
  31. 2024TOAST: Transformer Optimization using Adaptive and Simple Transformationscited by 1
  32. 2025Topology-Aware CLIP Few-Shot Learningcited by 1
  33. 2025Latent Manifold Reconstruction and Representation with Topological and Geometrical Regularizationcited by 1
  34. 2025Uncovering the Computational Ingredients of Human-Like Representations in LLMscited by 1
  35. 2026TopoPrune: Robust Data Pruning via Unified Latent Space Topologycited by 1
  36. 2026Towards Understanding Multimodal Fine-Tuning: Spatial Featurescited by 1
  37. 2026The Density of Cross-Persistence Diagrams and Its Applicationscited by 1
  38. 2026Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learningcited by 1

50 most cited of 66 All 66 (JSON)