# Alignment and stability of embeddings: measurement and inference improvement

arXiv: https://arxiv.org/abs/2101.07251 · HTML: https://mlsanity.com/paper/2101.07251/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 20 of 20)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [Representation Learning for Dynamic Graphs: A Survey](https://mlsanity.com/paper/1905.11485/) (2019)
- [A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications](https://mlsanity.com/paper/1709.07604/) (2017)
- [Graph Embedding Techniques, Applications, and Performance: A Survey](https://mlsanity.com/paper/1705.02801/) (2017)
- [A Survey on Network Embedding](https://mlsanity.com/paper/1711.08752/) (2017)
- [DynGEM: Deep Embedding Method for Dynamic Graphs](https://mlsanity.com/paper/1805.11273/) (2018)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [Contact patterns in a high school: a comparison between data collected using wearable sensors, contact diaries and friendship surveys](https://mlsanity.com/paper/1506.03645/) (2015)
- [A Survey on Embedding Dynamic Graphs](https://mlsanity.com/paper/2101.01229/) (2021)
- [Node Embedding over Temporal Graphs](https://mlsanity.com/paper/1903.08889/) (2019)
- [dynnode2vec: Scalable Dynamic Network Embedding](https://mlsanity.com/paper/1812.02356/) (2018)
- [A Comparative Study for Unsupervised Network Representation Learning](https://mlsanity.com/paper/1903.07902/) (2019)
- [GloDyNE: Global Topology Preserving Dynamic Network Embedding](https://mlsanity.com/paper/2008.01935/) (2020)
- [Dynamic Network Embeddings for Network Evolution Analysis](https://mlsanity.com/paper/1906.09860/) (2019)
- [Link prediction in dynamic networks using random dot product graphs](https://mlsanity.com/paper/1912.10419/) (2019)
- [Change Detection in Noisy Dynamic Networks: A Spectral Embedding Approach](https://mlsanity.com/paper/1910.02301/) (2019)

## Built on by (most cited first, 4 of 4)

- [I0T: Embedding Standardization Method Towards Zero Modality Gap](https://mlsanity.com/paper/2412.14384/) (2024)
- [Geometric instability of graph neural networks on large graphs](https://mlsanity.com/paper/2308.10099/) (2023)
- [Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs](https://mlsanity.com/paper/2605.12714/) (2026)
- [Investigating internal migration with network analysis and latent space representations: An application to Turkey](https://mlsanity.com/paper/2201.03543/) (2022)

## Abstract (arXiv)

Representation learning (RL) methods learn objects' latent embeddings where information is preserved by distances. Since distances are invariant to certain linear transformations, one may obtain different embeddings while preserving the same information. In dynamic systems, a temporal difference in embeddings may be explained by the stability of the system or by the misalignment of embeddings due to arbitrary transformations. In the literature, embedding alignment has not been defined formally, explored theoretically, or analyzed empirically. Here, we explore the embedding alignment and its parts, provide the first formal definitions, propose novel metrics to measure alignment and stability, and show their suitability through synthetic experiments. Real-world experiments show that both static and dynamic RL methods are prone to produce misaligned embeddings and such misalignment worsens the performance of dynamic network inference tasks. By ensuring alignment, the prediction accuracy raises by up to 90% in static and by 40% in dynamic RL methods.