mlsanityData to 14 Oct 2026

A Simple and Plug-and-play Method for Unsupervised Sentence Representation Enhancement

arXiv 2305.07824 ↗First posted 13 May 2023cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Generating proper embedding of sentences through an unsupervised way is beneficial to semantic matching and retrieval problems in real-world scenarios. This paper presents Representation ALchemy (RepAL), an extremely simple post-processing method that enhances sentence representations. The basic idea in RepAL is to de-emphasize redundant information of sentence embedding generated by pre-trained models. Through comprehensive experiments, we show that RepAL is free of training and is a plug-and-play method that can be combined with most existing unsupervised sentence learning models. We also conducted in-depth analysis to understand RepAL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 13 May 2023.

Builds on

22 of 34 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  7. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  8. 2014Distributed Representations of Sentences and Documentscited by 678
  9. 2015Skip-Thought Vectorscited by 322
  10. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  11. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  12. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
10 more
  1. 2018An efficient framework for learning sentence representationscited by 198
  2. 2019End-to-End Bias Mitigation by Modelling Biases in Corporacited by 128
  3. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  4. 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 104
  5. 2016Learning Distributed Representations of Sentences from Unlabelled Datacited by 91
  6. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  7. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
  8. 2021CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understandingcited by 50
  9. 2019NEZHA: Neural Contextualized Representation for Chinese Language Understandingcited by 39
  10. 2020Towards Robustifying NLI Models Against Lexical Dataset Biasescited by 34

Built on by

Not cited yet.