mlsanityData to 7 Oct 2026

Retrofitting Multilingual Sentence Embeddings with Abstract Meaning Representation

arXiv 2210.09773 ↗First posted 18 Oct 2022cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We introduce a new method to improve existing multilingual sentence embeddings with Abstract Meaning Representation (AMR). Compared with the original textual input, AMR is a structured semantic representation that presents the core concepts and relations in a sentence explicitly and unambiguously. It also helps reduce surface variations across different expressions and languages. Unlike most prior work that only evaluates the ability to measure semantic similarity, we present a thorough evaluation of existing multilingual sentence embeddings and our improved versions, which include a collection of five transfer tasks in different downstream applications. Experiment results show that retrofitting multilingual sentence embeddings with AMR leads to better state-of-the-art performance on both semantic textual similarity and transfer tasks. Our codebase and evaluation scripts can be found at urlhttps://github.com/jcyk/MSE-AMR.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 71 of its 100 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

71 of 100 references.

References · 71

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,837
38 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  2. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  3. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  4. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  5. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  6. 2014Distributed Representations of Sentences and Documentscited by 677
  7. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  8. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  11. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  12. 2015Skip-Thought Vectorscited by 321
  13. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  14. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 291
  15. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  16. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  17. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 262
  18. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  19. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  20. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  21. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 221
  22. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 198
  23. 2018An efficient framework for learning sentence representationscited by 197
  24. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 192
  25. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  26. 2020The Multilingual Amazon Reviews Corpuscited by 142
  27. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  28. 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 103
  29. 2021Self-Guided Contrastive Learning for BERT Sentence Representationscited by 103
  30. 2020An Unsupervised Sentence Embedding Method by Mutual Information Maximizationcited by 91
  31. 2016Learning Distributed Representations of Sentences from Unlabelled Datacited by 90
  32. 2019Graph Transformer for Graph-to-Sequence Learningcited by 82
  33. 2017On Sampling Strategies for Neural Network-based Collaborative Filteringcited by 71
  34. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 68
  35. 2018Graph-to-Sequence Learning using Gated Graph Neural Networkscited by 61
  36. 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
  37. 2022Graph Pre-training for AMR Parsing and Generationcited by 55
  38. 2017Learning Joint Multilingual Sentence Representations with Neural Machine Translationcited by 52

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1.

4 papers.

  1. 2023mPMR: A Multilingual Pre-trained Machine Reader at Scalecited by 2
  2. 2023Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learningcited by 2
  3. 2023Not Enough Labeled Data? Just Add Semantics: A Data-Efficient Method for Inferring Online Health Textscited by 1
  4. 2024Cross-lingual Contextualized Phrase Retrievalcited by 1