Do Multilingual Encoders Produce Language-Consistent Semantic IDs?
Not on the map. Why?
Abstract (arXiv)
Semantic IDs (SIDs) compress item embeddings into discrete code sequences used in generative retrieval. We ask whether a multilingual encoder is sufficient for different-language renderings of the same product to receive language-consistent SIDs. Using Amazon ESCI listings rendered in English, Spanish, and Japanese, we test whether translations remain close to their English source, whether residual quantization is unusually sensitive to translation-induced movement, and whether multilingual or language-balanced quantizer fitting improves SID agreement. Multilingual E5 places translations measurably apart: under an English-heavy fit, a Japanese translation preserves the first SID code of its English counterpart in only 7.7% of cases, compared with 89.0% for an English rewording. Distance-matched product-directed controls produce nearly the same full-SID mismatch as translation, providing no evidence that the quantizer selectively amplifies language directions. Balancing the fitting mixture makes codebook use more uniform but further reduces cross-lingual prefix agreement: Spanish first-code consistency falls from 28.3% to 6.6%, while an English-only fit preserves it for 67.6% of Spanish translations. These results show that multilingual exposure and balanced codebook use alone do not guarantee language-consistent SIDs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 16 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
8 of 16 references.
References · 8
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
- 2022Autoregressive Image Generation using Residual Quantizationcited by 722
- 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
- 2022Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Searchcited by 65
- 2021Cross-Market Product Recommendationcited by 22
- 2025Multilingual Generative Retrieval via Cross-lingual Semantic Compressioncited by 4
- 2026Decoupled Residual Quantization for Robust Semantic IDs in Recommendationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation
- 2024Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval
- 2024Understanding Cross-Lingual Alignment -- A Survey
- 2025Evaluating Multilingual and Code-Switched Alignment in LLMs via Synthetic Natural Language Inference
- 2025How Language Directions Align with Token Geometry in Multilingual LLMs
- 2026LLMs Need Encoders for Semantic IDs Too
- 2026Lingua Franca or Probing Artifact? Rethinking Latent Language in Multilingual LLMs
- 2025Benchmarking Cross-Lingual Semantic Alignment in Multilingual Embeddings