mlsanityData to 7 Oct 2026

Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks

arXiv 2010.08240 ↗First posted 16 Oct 2020cs.CLcited by 136

Not on the map. Why?

Abstract (arXiv)

There are two approaches for pairwise sentence scoring: Cross-encoders, which perform full-attention over the input pair, and Bi-encoders, which map each input independently to a dense vector space. While cross-encoders often achieve higher performance, they are too slow for many practical use cases. Bi-encoders, on the other hand, require substantial training data and fine-tuning over the target task to achieve competitive performance. We present a simple yet efficient data augmentation strategy called Augmented SBERT, where we use the cross-encoder to label a larger set of input pairs to augment the training data for the bi-encoder. We show that, in this process, selecting the sentence pairs is non-trivial and crucial for the success of the method. We evaluate our approach on multiple tasks (in-domain) as well as on a domain adaptation task. Augmented SBERT achieves an improvement of up to 6 points for in-domain and of up to 37 points for domain adaptation tasks compared to the original bi-encoder performance.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 25; 2023: 27; 2024: 28; 2025: 26; 2026: 8.

136 papers.

Sort:

By area:

  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  2. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 779
  3. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  4. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  5. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  6. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  7. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  8. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  9. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  10. 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
  11. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  12. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 68
38 more
  1. 2024GuardT2I: Defending Text-to-Image Models from Adversarial Promptscited by 61
  2. 2023UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankerscited by 37
  3. 2021The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Designcited by 33
  4. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  5. 2023Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Costcited by 33
  6. 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
  7. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  8. 2021Improving Unsupervised Dialogue Topic Segmentation with Utterance-Pair Coherence Scoringcited by 29
  9. 2023Improving Contrastive Learning of Sentence Embeddings from AI Feedbackcited by 29
  10. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23
  11. 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspectivecited by 23
  12. 2021PatentSBERTa: A Deep NLP based Hybrid Model for Patent Distance and Classification using Augmented SBERTcited by 22
  13. 2021On Cross-Lingual Retrieval with Multilingual Text Encoderscited by 22
  14. 2022A Thorough Examination on Zero-shot Dense Retrievalcited by 21
  15. 2021Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillationscited by 20
  16. 2021Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violencecited by 19
  17. 2021Question Answering Infused Pre-training of General-Purpose Contextualized Representationscited by 19
  18. 2022Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Frameworkcited by 18
  19. 2024MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answeringcited by 17
  20. 2022Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Databasecited by 16
  21. 2022Ranking-Enhanced Unsupervised Sentence Representation Learningcited by 15
  22. 2023Targeted Data Generation: Finding and Fixing Model Weaknessescited by 15
  23. 2024SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysiscited by 13
  24. 2024Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-trainingcited by 13
  25. 2022Multimedia Generative Script Learning for Task Planningcited by 12
  26. 2023MGeo: Multi-Modal Geographic Pre-Training Methodcited by 12
  27. 2024Exploring the Capabilities of Prompted Large Language Models in Educational and Assessment Applicationscited by 12
  28. 2023A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the ChatGPT Era and Beyondcited by 11
  29. 2022Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorizationcited by 10
  30. 2025Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objectscited by 10
  31. 2022CrowdChecked: Detecting Previously Fact-Checked Claims in Social Mediacited by 9
  32. 2022NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialoguecited by 8
  33. 2023Verifying the Robustness of Automatic Credibility Assessmentcited by 8
  34. 2024A Survey of Data Synthesis Approachescited by 8
  35. 2022The Dialog Must Go On: Improving Visual Dialog via Generative Self-Trainingcited by 7
  36. 2022Improving Bilingual Lexicon Induction with Cross-Encoder Rerankingcited by 7
  37. 2023Short Answer Grading Using One-shot Prompting and Text Similarity Scoring Modelcited by 7
  38. 2023Learning Human-like Representations to Enable Learning Human Valuescited by 7

50 most cited of 136 All 136 (JSON)