CLIP also Understands Text: Prompting CLIP for Phrase Understanding
Not on the map. Why?
Abstract (arXiv)
Contrastive Language-Image Pretraining (CLIP) efficiently learns visual concepts by pre-training with natural language supervision. CLIP and its visual encoder have been explored on various vision and language tasks and achieve strong zero-shot or transfer learning performance. However, the application of its text encoder solely for text understanding has been less explored. In this paper, we find that the text encoder of CLIP actually demonstrates strong ability for phrase understanding, and can even significantly outperform popular language models such as BERT with a properly designed prompt. Extensive experiments validate the effectiveness of our method across different datasets and domains on entity clustering and entity set expansion tasks.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 11 Oct 2022.
Builds on
22 of 29 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
- 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
- 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
- 2021How Many Data Points is a Prompt Worth?cited by 198
10 more
- 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 183
- 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
- 2020Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervisioncited by 94
- 2020Learning Dense Representations of Phrases at Scalecited by 91
- 2021Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Explorationcited by 44
- 2017Self-Taught Convolutional Neural Networks for Short Text Clusteringcited by 34
- 2022UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Miningcited by 33
- 2019SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemblecited by 25
- 2017Learning Phrase Embeddings from Paraphrases with GRUscited by 4
- 2022XDBERT: Distilling Visual Information to BERT from Cross-Modal Systems to Improve Language Understandingcited by 2
Built on by
Citing papers per year: 2023: 3.
3 papers.