mlsanityData to 14 Oct 2026

Neural Embeddings for Text

arXiv 2208.08386 ↗First posted 17 Aug 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose a new kind of embedding for natural language text that deeply represents semantic meaning. Standard text embeddings use the outputs from hidden layers of a pretrained language model. In our method, we let a language model learn from the text and then literally pick its brain, taking the actual weights of the model's neurons to generate a vector. We call this representation of the text a neural embedding. We confirm the ability of this representation to reflect semantics of the text by an analysis of its behavior on several datasets, and by a comparison of neural embedding with state of the art sentence embeddings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 17 Aug 2022.

Builds on

22 of 32 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  4. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  5. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  6. 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
  7. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  8. 2014Deep metric learning using Triplet networkcited by 422
  9. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
  10. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  11. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  12. 2014Learning Fine-grained Image Similarity with Deep Rankingcited by 190
10 more
  1. 2022DiffCSE: Difference-based Contrastive Learning for Sentence Embeddingscited by 132
  2. 2022Should You Mask 15% in Masked Language Modeling?cited by 118
  3. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  4. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  5. 2020Fill in the BLANC: Human-free quality estimation of document summariescited by 82
  6. 2021Pairwise Supervised Contrastive Learning of Sentence Representationscited by 31
  7. 2022Quality Controlled Paraphrase Generationcited by 31
  8. 2021Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Modelscited by 22
  9. 2020Is human scoring the best criteria for summary evaluation?cited by 7
  10. 2022Masked Autoencoders As The Unified Learners For Pre-Trained Sentence Representationcited by 5

Built on by

Not cited yet.