mlsanityData to 7 Oct 2026

Optimal synthesis embeddings

arXiv 2406.10259 ↗First posted 10 Jun 2024cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we introduce a word embedding composition method based on the intuitive idea that a fair embedding representation for a given set of words should satisfy that the new vector will be at the same distance of the vector representation of each of its constituents, and this distance should be minimized. The embedding composition method can work with static and contextualized word representations, it can be applied to create representations of sentences and learn also representations of sets of words that are not necessarily organized as a sequence. We theoretically characterize the conditions for the existence of this type of representation and derive the solution. We evaluate the method in data augmentation and sentence classification tasks, investigating several design choices of embeddings and composition methods. We show that our approach excels in solving probing tasks designed to capture simple linguistic features of sentences.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 37 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

18 of 37 references.

References · 18

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  2. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  5. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  6. 2018Universal Sentence Encodercited by 485
  7. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  8. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  9. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  10. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  11. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  12. 2016Diachronic Word Embeddings Reveal Statistical Laws of Semantic Changecited by 219
6 more
  1. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  2. 2019Analogies Explained: Towards Understanding Word Embeddingscited by 63
  3. 2015Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learningcited by 29
  4. 2017Semantic Structure and Interpretability of Word Embeddingscited by 21
  5. 2019Analytical Methods for Interpretable Ultradense Word Embeddingscited by 13
  6. 2016Ultradense Word Embeddings by Orthogonal Transformationcited by 8

Built on by

Not cited yet.