# Exploring Intra and Inter-language Consistency in Embeddings with ICA

arXiv: https://arxiv.org/abs/2406.12474 · HTML: https://mlsanity.com/paper/2406.12474/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 18 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 4 of 4)

- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Discovering Universal Geometry in Embeddings with ICA](https://mlsanity.com/paper/2305.13175/) (2023)

## Built on by (most cited first, 2 of 2)

- [Transforming Hidden States into Binary Semantic Features](https://mlsanity.com/paper/2409.19813/) (2024)
- [Structure Retention in Embedding Spaces as a Predictor of Benchmark Performance](https://mlsanity.com/paper/2605.22202/) (2026)

## Abstract (arXiv)

Word embeddings represent words as multidimensional real vectors, facilitating data analysis and processing, but are often challenging to interpret. Independent Component Analysis (ICA) creates clearer semantic axes by identifying independent key features. Previous research has shown ICA's potential to reveal universal semantic axes across languages. However, it lacked verification of the consistency of independent components within and across languages. We investigated the consistency of semantic axes in two ways: both within a single language and across multiple languages. We first probed into intra-language consistency, focusing on the reproducibility of axes by performing ICA multiple times and clustering the outcomes. Then, we statistically examined inter-language consistency by verifying those axes' correspondences using statistical tests. We newly applied statistical methods to establish a robust framework that ensures the reliability and universality of semantic axes.