mlsanityData to 14 Oct 2026

Ask2Transformers: Zero-Shot Domain labelling with Pre-trained Language Models

arXiv 2101.02661 ↗First posted 7 Jan 2021cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

In this paper we present a system that exploits different pre-trained Language Models for assigning domain labels to WordNet synsets without any kind of supervision. Furthermore, the system is not restricted to use a particular set of domain labels. We exploit the knowledge encoded within different off-the-shelf pre-trained Language Models and task formulations to infer the domain label of a particular WordNet definition. The proposed zero-shot system achieves a new state-of-the-art on the English dataset used in the evaluation.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 12 of its 23 references that are in the map. First posted on arXiv 7 Jan 2021.

Builds on

12 of 23 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  8. 2018Deep contextualized word representationscited by 2,290
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  11. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 308
  12. 2019Denoising based Sequence-to-Sequence Pre-training for Text Generationcited by 15

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 6; 2023: 1; 2024: 2; 2025: 1.

13 papers.

Sort:

By area:

  1. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  2. 2021Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extractioncited by 73
  3. 2021Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarizationcited by 46
  4. 2022Zero-Shot Text Classification with Self-Trainingcited by 26
  5. 2022Textual Entailment for Event Argument Extraction: Zero- and Few-Shot with Multi-Source Learningcited by 25
  6. 2023What do Language Models know about word senses? Zero-Shot WSD with Language Models and Domain Inventoriescited by 9
  7. 2022Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Inductioncited by 8
  8. 2022ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizationscited by 7
  9. 2022Template-based Approach to Zero-shot Intent Recognitioncited by 5
  10. 2024Pointing out the Shortcomings of Relation Extraction Models with Semantically Motivated Adversarialscited by 1
  11. 2024Text2Traj2Text: Learning-by-Synthesis Framework for Contextual Captioning of Human Movement Trajectoriescited by 1
  12. 2022Why Do Neural Language Models Still Need Commonsense Knowledge to Handle Semantic Variations in Question Answering?cited by 0
1 more
  1. 2025Medical Argument Mining: Exploitation of Scarce Data Using NLI Systemscited by 0