mlsanityData to 14 Oct 2026

Getting BART to Ride the Idiomatic Train: Learning to Represent Idiomatic Expressions

arXiv 2207.03679 ↗First posted 8 Jul 2022cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Idiomatic expressions (IEs), characterized by their non-compositionality, are an important part of natural language. They have been a classical challenge to NLP, including pre-trained language models that drive today's state-of-the-art. Prior work has identified deficiencies in their contextualized representation stemming from the underlying compositional paradigm of representation. In this work, we take a first-principles approach to build idiomaticity into BART using an adapter as a lightweight non-compositional language expert trained on idiomatic sentences. The improved capability over baselines (e.g., BART) is seen via intrinsic and extrinsic methods, where idiom embeddings score 0.19 points higher in homogeneity score for embedding clustering, and up to 25% higher sequence accuracy on the idiom processing tasks of IE sense disambiguation and span detection.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 30 of its 72 references that are in the map. First posted on arXiv 8 Jul 2022.

Builds on

30 of 72 references.

References · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  7. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  8. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  9. 2017Learning multiple visual domains with residual adapterscited by 557
  10. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  11. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  12. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
18 more
  1. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  2. 2018Efficient parametrization of multi-domain deep neural networkscited by 168
  3. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  4. 2019Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIscited by 58
  5. 2021Dict-BERT: Enhancing Language Model Pre-training with Dictionarycited by 38
  6. 2019Evaluating Word Embedding Models: Methods and Experimental Resultscited by 37
  7. 2018Examining the Tip of the Iceberg: A Data Set for Idiom Translationcited by 34
  8. 2021AStitchInLanguageModels: Dataset and Methods for the Exploration of Idiomaticity in Pre-Trained Language Modelscited by 27
  9. 2021Figurative Language in Recognizing Textual Entailmentcited by 26
  10. 2021Investigating Robustness of Dialog Models to Popular Figurative Language Constructscited by 19
  11. 2021Idiomatic Expression Identification using Semantic Compatibilitycited by 19
  12. 2020Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extractioncited by 18
  13. 2021Potential Idiomatic Expression (PIE)-English: Corpus for Classes of Idiomscited by 15
  14. 2018Classifying Idiomatic and Literal Expressions Using Topic Models and Intensity of Emotionscited by 13
  15. 2016Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddingscited by 9
  16. 2020MICE: Mining Idioms with Contextual Embeddingscited by 7
  17. 2021Idiomatic Expression Paraphrasing without Strong Supervisioncited by 7
  18. 2019On the Compositionality Prediction of Noun Phrases using Poincar'e Embeddingscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 1.

4 papers.

  1. 2024Semantics of Multiword Expressions in Transformer-Based Models: A Surveycited by 9
  2. 2025Evaluating LLMs on Chinese Idiom Translationcited by 2
  3. 2024Investigating Idiomaticity in Word Representationscited by 1
  4. 2023Unsupervised Paraphrasing of Multiword Expressionscited by 0