What do Deck Chairs and Sun Hats Have in Common? Uncovering Shared Properties in Large Concept Vocabularies
Not on the map. Why?
Abstract (arXiv)
Concepts play a central role in many applications. This includes settings where concepts have to be modelled in the absence of sentence context. Previous work has therefore focused on distilling decontextualised concept embeddings from language models. But concepts can be modelled from different perspectives, whereas concept embeddings typically mostly capture taxonomic structure. To address this issue, we propose a strategy for identifying what different concepts, from a potentially large concept vocabulary, have in common with others. We then represent concepts in terms of the properties they share with the other concepts. To demonstrate the practical usefulness of this way of modelling concepts, we consider the task of ultra-fine entity typing, which is a challenging multi-label classification problem. We show that by augmenting the label set with shared properties, we can improve the performance of the state-of-the-art models for this task.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 23 Oct 2023.
Builds on
27 of 43 references.
References · 27
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,239
- 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 774
- 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
- 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
- 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 201
- 2020Probing Pretrained Language Models for Lexical Semanticscited by 156
- 2019Adaptive Cross-Modal Few-Shot Learningcited by 95
- 2021Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoderscited by 88
- 2018Ultra-Fine Entity Typingcited by 87
- 2020Boosting Few-Shot Learning With Adaptive Margin Losscited by 83
15 more
- 2020Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Networkcited by 83
- 2018Attentive Relational Networks for Mapping Images to Scene Graphscited by 73
- 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
- 2020GenericsKB: A Knowledge Base of Generic Statementscited by 51
- 2019Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typingcited by 22
- 2021Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attentioncited by 15
- 2022Automatic Noisy Label Correction for Fine-Grained Entity Typingcited by 13
- 2021Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learningcited by 12
- 2021ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns' Semantic Properties and their Prototypicalitycited by 12
- 2020SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discoverycited by 10
- 2022Modelling Commonsense Properties using Pre-Trained Bi-Encoderscited by 9
- 2023Distilling Semantic Concept Embeddings from Contrastively Fine-Tuned Language Modelscited by 8
- 2023Ultra-Fine Entity Typing with Prior Knowledge about Labels: A Simple Clustering Based Strategycited by 8
- 2020Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddingscited by 6
- 2016Automated Generation of Multilingual Clusters for the Evaluation of Distributed Representationscited by 3
Built on by
Citing papers per year: 2024: 1.
1 paper.