mlsanityData to 7 Oct 2026

Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing

arXiv 2211.02483 ↗First posted 4 Nov 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

The explosion of e-commerce has caused the need for processing and analysis of product titles, like entity typing in product titles. However, the rapid activity in e-commerce has led to the rapid emergence of new entities, which is difficult to be solved by general entity typing. Besides, product titles in e-commerce have very different language styles from text data in general domain. In order to handle new entities in product titles and address the special language styles problem of product titles in e-commerce domain, we propose our textual entailment model with continuous prompt tuning based hypotheses and fusion embeddings for e-commerce entity typing. First, we reformulate the entity typing task into a textual entailment problem to handle new entities that are not present during training. Second, we design a model to automatically generate textual entailment hypotheses using a continuous prompt tuning method, which can generate better textual entailment hypotheses without manual design. Third, we utilize the fusion embeddings of BERT embedding and CharacterBERT embedding with a two-layer MLP classifier to solve the problem that the language styles of product titles in e-commerce are different from that of general domain. To analyze the effect of each contribution, we compare the performance of entity typing and textual entailment model, and conduct ablation studies on continuous prompt tuning and fusion embeddings. We also evaluate the impact of different prompt template initialization for the continuous prompt tuning. We show our proposed model improves the average F1 score by around 2% compared to the baseline BERT entity typing model.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 4 Nov 2022.

Builds on

31 of 39 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  10. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
19 more
  1. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  2. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 299
  3. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  4. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  5. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  6. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  7. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  8. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  9. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  10. 2015Character-Aware Neural Language Modelscited by 129
  11. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  12. 2020CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characterscited by 87
  13. 2020CharBERT: Character-aware Pre-trained Language Modelcited by 63
  14. 2021Prompt-Learning for Fine-Grained Entity Typingcited by 62
  15. 2022Continual Prompt Tuning for Dialog State Trackingcited by 44
  16. 2022Prompt Tuning for Discriminative Pre-trained Language Modelscited by 17
  17. 2016Character-Level Language Modeling with Hierarchical Recurrent Neural Networkscited by 13
  18. 2022Are Prompt-based Models Clueless?cited by 11
  19. 2021Continuous Entailment Patterns for Lexical Inference in Contextcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 1.

5 papers.

  1. 2023Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuningcited by 15
  2. 2024A survey on fairness of large language models in e-commerce: progress, application, and challengecited by 10
  3. 2023PromptAttack: Probing Dialogue State Trackers with Adversarial Promptscited by 6
  4. 2023Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tablescited by 6
  5. 2023Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approachcited by 2