mlsanityData to 7 Oct 2026

Normalized Space Alignment: A Versatile Metric for Representation Analysis

arXiv 2411.04512 ↗First posted 7 Nov 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a manifold analysis technique for neural network representations. Normalized Space Alignment (NSA) compares pairwise distances between two point clouds derived from the same source and having the same size, while potentially possessing differing dimensionalities. NSA can act as both an analytical tool and a differentiable loss function, providing a robust means of comparing and aligning representations across different layers and models. It satisfies the criteria necessary for both a similarity metric and a neural network loss function. We showcase NSA's versatility by illustrating its utility as a representation space analysis metric, a structure-preserving loss function, and a robustness analysis tool. NSA is not only computationally efficient but it can also approximate the global structural discrepancy during mini-batching, facilitating its use in a wide variety of neural network training paradigms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 88 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

41 of 88 references.

References · 41

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  10. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  11. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  12. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
29 more
  1. 2020Graph Structure Learning for Robust Graph Neural Networkscited by 368
  2. 2018Insights on representational similarity in neural networks with canonical correlationcited by 318
  3. 2021GRAND: Graph Neural Diffusioncited by 258
  4. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
  5. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 215
  6. 2019Topology Attack and Defense for Graph Neural Networks: An Optimization Perspectivecited by 196
  7. 2020GNNGuard: Defending Graph Neural Networks against Adversarial Attackscited by 195
  8. 2020Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualizationcited by 170
  9. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  10. 2019Topological Autoencoderscited by 124
  11. 2021Generalized Shape Metrics on Neural Representationscited by 109
  12. 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
  13. 2016Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Changecited by 63
  14. 2018Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representationcited by 62
  15. 2018Geometry Score: A Method For Comparing Generative Adversarial Networkscited by 56
  16. 2020Adversarial Attacks and Defenses on Graphs: A Review, A Tool and Empirical Studiescited by 42
  17. 2019The Shape of Data: Intrinsic Distance for Data Distributionscited by 38
  18. 2023Are Defenses for Graph Neural Networks Robust?cited by 38
  19. 2023Learning Topology-Preserving Data Representationscited by 30
  20. 2023Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principlescited by 22
  21. 2022GULP: a prediction-based metric between representationscited by 17
  22. 2020Similarity of Neural Networks with Gradientscited by 16
  23. 2021POLE: Polarized Embedding for Signed Networkscited by 16
  24. 2021A Broader Picture of Random-walk Based Graph Embeddingcited by 15
  25. 2020The Effects of Randomness on the Stability of Node Embeddingscited by 14
  26. 2019On the Downstream Performance of Compressed Word Embeddingscited by 13
  27. 2023Representation Learning via Manifold Flattening and Reconstructioncited by 11
  28. 2021Graph-Based Similarity of Neural Network Representationscited by 7
  29. 2015A Distance Measure for the Analysis of Polar Opinion Dynamics in Social Networkscited by 3

Built on by

Not cited yet.