mlsanityData to 14 Oct 2026

Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training

arXiv 2405.06932 ↗First posted 11 May 2024cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

In this report, we introduce Piccolo2, an embedding model that surpasses other models in the comprehensive evaluation over 6 tasks on CMTEB benchmark, setting a new state-of-the-art. Piccolo2 primarily leverages an efficient multi-task hybrid loss training approach, effectively harnessing textual data and labels from diverse downstream tasks. In addition, Piccolo2 scales up the embedding dimension and uses MRL training to support more flexible vector dimensions. The latest information of piccolo models can be accessed via: https://huggingface.co/sensenova/

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 11 May 2024.

Builds on

20 of 33 references.

References · 20

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  7. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  9. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  10. 2023Improving Text Embeddings with Large Language Modelscited by 303
  11. 2022Matryoshka Representation Learningcited by 282
  12. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 278
8 more
  1. 2020CORD-19: The COVID-19 Open Research Datasetcited by 180
  2. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  3. 2020OCNLI: Original Chinese Natural Language Inferencecited by 86
  4. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 83
  5. 2023T2Ranking: A large-scale Chinese Benchmark for Passage Rankingcited by 60
  6. 2022CSL: A Large-scale Chinese Scientific Literature Datasetcited by 45
  7. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 27
  8. 2022DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Enginecited by 15

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 7.

10 papers.

  1. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  2. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 36
  3. 2025Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representationcited by 26
  4. 2024CoFE-RAG: A Comprehensive Full-chain Evaluation Framework for Retrieval-Augmented Generation with Enhanced Data Diversitycited by 11
  5. 2025GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Trainingcited by 5
  6. 2025PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text Embeddingcited by 4
  7. 2025SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generationcited by 3
  8. 2025QZhou-Embedding Technical Reportcited by 3
  9. 2025LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBaycited by 2
  10. 2024Can Machines Resonate with Humans? Evaluating the Emotional and Empathic Comprehension of LMscited by 1