mlsanityData to 14 Oct 2026

FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

arXiv 2604.18109 ↗First posted 20 Apr 2026cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover the lexical content from multilingual (LaBSE), multimodal (SONAR) and API-based (Gemini) sentence embedding spaces in several high- and mid-resource languages. We show that FLiP can recall more than 75% of lexical content from the embeddings, significantly outperforming existing non-factorized baselines. Using this as a diagnostic tool, we uncover the modality and language biases across the selected sentence encoders and provide practitioners with intrinsic insights about the encoders without relying on conventional downstream evaluation tasks. Our implementation is public https://github.com/BUTSpeechFIT/FLiP.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 25 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

20 of 25 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  6. 2014Distributed Representations of Sentences and Documentscited by 678
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  8. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  9. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  10. 2017Implicit Regularization in Matrix Factorizationcited by 322
  11. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  12. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
8 more
  1. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
  2. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  3. 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
  4. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  5. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  6. 2025SENSE models: an open source solution for multilingual and multimodal semantic-based taskscited by 6
  7. 2025Transformation of audio embeddings into interpretable, concept-based representationscited by 5
  8. 2025End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Datacited by 1

Built on by

Not cited yet.