# Open Sentence Embeddings for Portuguese with the Serafim PT* encoders family

arXiv: https://arxiv.org/abs/2407.19527 · HTML: https://mlsanity.com/paper/2407.19527/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 28 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning](https://mlsanity.com/paper/2104.06979/) (2021)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [AnglE-optimized Text Embeddings](https://mlsanity.com/paper/2309.12871/) (2023)
- [GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning](https://mlsanity.com/paper/2402.16829/) (2024)
- [Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE](https://mlsanity.com/paper/2212.01853/) (2022)
- [Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*](https://mlsanity.com/paper/2305.06721/) (2023)
- [Advancing Generative AI for Portuguese with Open Decoder Gerv'asio PT*](https://mlsanity.com/paper/2402.18766/) (2024)
- [Fostering the Ecosystem of Open Neural Encoders for Portuguese with Albertina PT* Family](https://mlsanity.com/paper/2403.01897/) (2024)
- [PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese](https://mlsanity.com/paper/2404.05333/) (2024)

## Built on by (most cited first, 4 of 4)

- [Sovereign AI-based Public Services are Viable and Affordable](https://mlsanity.com/paper/2603.01869/) (2026)
- [Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders](https://mlsanity.com/paper/2607.04071/) (2026)
- [MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese](https://mlsanity.com/paper/2607.04581/) (2026)
- [Language Models for Portuguese: A Systematic Mapping Study](https://mlsanity.com/paper/2608.18138/) (2026)

## Abstract (arXiv)

Sentence encoder encode the semantics of their input, enabling key downstream applications such as classification, clustering, or retrieval. In this paper, we present Serafim PT*, a family of open-source sentence encoders for Portuguese with various sizes, suited to different hardware/compute budgets. Each model exhibits state-of-the-art performance and is made openly available under a permissive license, allowing its use for both commercial and research purposes. Besides the sentence encoders, this paper contributes a systematic study and lessons learned concerning the selection criteria of learning objectives and parameters that support top-performing encoders.