mlsanityData to 14 Oct 2026

Discriminating Form and Meaning in Multilingual Models with Minimal-Pair ABX Tasks

arXiv 2505.17747 ↗First posted 23 May 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a set of training-free ABX-style discrimination tasks to evaluate how multilingual language models represent language identity (form) and semantic content (meaning). Inspired from speech processing, these zero-shot tasks measure whether minimal differences in representation can be reliably detected. This offers a flexible and interpretable alternative to probing. Applied to XLM-R (Conneau et al, 2020) across pretraining checkpoints and layers, we find that language discrimination declines over training and becomes concentrated in lower layers, while meaning discrimination strengthens over time and stabilizes in deeper layers. We then explore probing tasks, showing some alignment between our metrics and linguistic learning performance. Our results position ABX tasks as a lightweight framework for analyzing the structure of multilingual representations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

39 of 52 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  7. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  8. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  9. 2019How multilingual is Multilingual BERT?cited by 645
  10. 2019Designing and Interpreting Probes with Control Taskscited by 640
  11. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  12. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
27 more
  1. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
  2. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  3. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 271
  4. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  5. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
  6. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 181
  7. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 146
  8. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 85
  9. 2024On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neuronscited by 83
  10. 2025WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialectscited by 79
  11. 2025Do Multilingual LLMs Think In English?cited by 78
  12. 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
  13. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  14. 2017The Zero Resource Speech Challenge 2017cited by 45
  15. 2024Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?cited by 41
  16. 2024Breaking the Curse of Multilinguality with Cross-lingual Expert Language Modelscited by 38
  17. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  18. 2022Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Modelscited by 32
  19. 2023Generative Spoken Language Model based on continuous word-sized audio tokenscited by 19
  20. 2022DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexiconcited by 12
  21. 2025Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMscited by 12
  22. 2022Evaluating context-invariance in unsupervised speech representationscited by 11
  23. 2025Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Modelscited by 11
  24. 2021On the Language-specificity of Multilingual BERT and the Impact of Fine-tuningcited by 8
  25. 2019Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Taskcited by 6
  26. 2022Is the Language Familiarity Effect gradual? A computational modelling approachcited by 2
  27. 2024Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignmentcited by 2

Built on by

Not cited yet.