mlsanityData to 7 Oct 2026

Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks

arXiv 2411.01192 ↗First posted 2 Nov 2024cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce bf Swan, a family of embedding models centred around the Arabic language, addressing both small-scale and large-scale use cases. Swan includes two variants: Swan-Small, based on ARBERTv2, and Swan-Large, built on ArMistral, a pretrained Arabic large language model. To evaluate these models, we propose ArabicMTEB, a comprehensive benchmark suite that assesses cross-lingual, multi-dialectal, multi-domain, and multi-cultural Arabic text embedding performance, covering eight diverse tasks and spanning 94 datasets. Swan-Large achieves state-of-the-art results, outperforming Multilingual-E5-large in most Arabic tasks, while the Swan-Small consistently surpasses Multilingual-E5-base. Our extensive evaluations demonstrate that Swan models are both dialectally and culturally aware, excelling across various Arabic domains while offering significant monetary efficiency. This work significantly advances the field of Arabic language modelling and provides valuable resources for future research and applications in Arabic natural language processing. Our models and benchmark are available at our GitHub page: hrefhttps://github.com/UBC-NLP/swanhttps://github.com/UBC-NLP/swan

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 40 of its 63 references that are in the map. First posted on arXiv 2 Nov 2024.

Builds on

40 of 63 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  9. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  10. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  11. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  12. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
28 more
  1. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  2. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  3. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  4. 2024Generative Representational Instruction Tuningcited by 227
  5. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 164
  6. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
  7. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  8. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  9. 2021The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Modelscited by 118
  10. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  11. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  12. 2022Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languagescited by 47
  13. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  14. 2018Integrating Stance Detection and Fact Checking in a Unified Corpuscited by 41
  15. 20161.5 billion words Arabic Corpuscited by 35
  16. 2021NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Taskcited by 32
  17. 2023NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Taskcited by 29
  18. 2024The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embeddingcited by 26
  19. 2022NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Taskcited by 23
  20. 2024NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Taskcited by 23
  21. 2023Dolphin: A Challenging and Diverse Benchmark for Arabic NLGcited by 22
  22. 2024Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddingscited by 20
  23. 2024PL-MTEB: Polish Massive Text Embedding Benchmarkcited by 19
  24. 2020Machine Generation and Detection of Arabic Manipulated and Fake Newscited by 16
  25. 2024Dallah: A Dialect-Aware Multimodal Large Language Model for Arabiccited by 16
  26. 2023Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classificationcited by 13
  27. 2024Enhancing Semantic Similarity Understanding in Arabic NLP with Nested Embedding Learningcited by 7
  28. 2022A Benchmark Study of Contrastive Learning for Arabic Social Meaningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 4.

10 papers.

  1. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  2. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  3. 2026From RAG to Agentic RAG for Faithful Islamic Question Answeringcited by 9
  4. 2025LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones?cited by 6
  5. 2025MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutchcited by 5
  6. 2025From Words to Proverbs: Evaluating LLMs Linguistic and Cultural Competence in Saudi Dialects with Abshercited by 2
  7. 2025CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Modelscited by 1
  8. 2026Linear Semantic Segmentation for Low-Resource Spoken Dialectscited by 1
  9. 2026SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptationcited by 0
  10. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0