mlsanityData to 14 Oct 2026

GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training

arXiv 2505.24581 ↗First posted 30 May 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Semantic textual similarity (STS) is a critical task in natural language processing (NLP), enabling applications in retrieval, clustering, and understanding semantic relationships between texts. However, research in this area for the Arabic language remains limited due to the lack of high-quality datasets and pre-trained models. This scarcity of resources has restricted the accurate evaluation and advance of semantic similarity in Arabic text. This paper introduces General Arabic Text Embedding (GATE) models that achieve state-of-the-art performance on the Semantic Textual Similarity task within the MTEB benchmark. GATE leverages Matryoshka Representation Learning and a hybrid loss training approach with Arabic triplet datasets for Natural Language Inference, which are essential for enhancing model performance in tasks that demand fine-grained semantic understanding. GATE outperforms larger models, including OpenAI, with a 20-25% performance improvement on STS benchmarks, effectively capturing the unique semantic nuances of Arabic.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 16 of its 34 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

16 of 34 references.

References · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  7. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  8. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  9. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  10. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  11. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  12. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
4 more
  1. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 166
  2. 2018Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Informationcited by 28
  3. 2024Piccolo2: General Text Embedding with Multi-task Hybrid Loss Trainingcited by 10
  4. 2024Enhancing Semantic Similarity Understanding in Arabic NLP with Nested Embedding Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  2. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  3. 2026ARCADE: A City-Scale Corpus for Fine-Grained Arabic Dialect Taggingcited by 1
  4. 2025Transformer Tafsir at QIAS 2025 Shared Task: Hybrid Retrieval-Augmented Generation for Islamic Knowledge Question Answeringcited by 0
  5. 2026What Makes a Good Fiqh Retriever? Answer Retrieval for Arabic Islamic Jurisprudencecited by 0