mlsanityData to 7 Oct 2026

Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations

arXiv 2005.08866 ↗First posted 18 May 2020cs.CLcited by 42

Not on the map. Why?

Abstract (arXiv)

We introduce Span-ConveRT, a light-weight model for dialog slot-filling which frames the task as a turn-based span extraction task. This formulation allows for a simple integration of conversational knowledge coded in large pretrained conversational models such as ConveRT (Henderson et al., 2019). We show that leveraging such knowledge in Span-ConveRT is especially useful for few-shot learning scenarios: we report consistent gains over 1) a span extractor that trains representations from scratch in the target domain, and 2) a BERT-based span extractor. In order to inspire more work on span extraction for the slot-filling task, we also release RESTAURANTS-8K, a new challenging data set of 8,198 utterances, compiled from actual conversations in the restaurant booking domain.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 16; 2023: 10; 2024: 6; 2025: 1.

42 papers.

Sort:

By area:

  1. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  2. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  3. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  4. 2022InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuningcited by 39
  5. 2022SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generationcited by 34
  6. 2022Learning to Execute Actions or Ask Clarification Questionscited by 30
  7. 2022SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understandingcited by 28
  8. 2021ConvFiT: Conversational Fine-Tuning of Pretrained Language Modelscited by 25
  9. 2023DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AIcited by 21
  10. 2022Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twittercited by 19
  11. 2022Label Semantic Aware Pre-training for Few-shot Text Classificationcited by 18
  12. 2022LAD: Language Models as Data for Zero-Shot Dialogcited by 18
30 more
  1. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  2. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  3. 2021GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Fillingcited by 11
  4. 2021Generative Conversational Networkscited by 11
  5. 2022Improved and Efficient Conversational Slot Labeling through Question Answeringcited by 10
  6. 2022UniDU: Towards A Unified Generative Dialogue Understanding Frameworkcited by 9
  7. 2022NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialoguecited by 8
  8. 2022Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialoguecited by 8
  9. 2023Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptationcited by 7
  10. 2021N-Shot Learning for Augmenting Task-Oriented Dialogue State Trackingcited by 6
  11. 2021Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domaincited by 6
  12. 2024A Survey of Ontology Expansion for Conversational Understandingcited by 6
  13. 2022Inverse is Better! Fast and Accurate Prompt for Few-shot Slot Taggingcited by 5
  14. 2023Transfer-Free Data-Efficient Multilingual Slot Labelingcited by 4
  15. 2023Overview of the CLAIMSCAN-2023: Uncovering Truth in Social Media through Claim Detection and Identification of Claim Spanscited by 4
  16. 2021SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teachingcited by 3
  17. 2022DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learningcited by 3
  18. 2022Semantic-based Pre-training for Dialogue Understandingcited by 3
  19. 2023Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Datacited by 3
  20. 2023UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Promptcited by 3
  21. 2024A Pointer Network-based Approach for Joint Extraction and Detection of Multi-Label Multi-Class Intentscited by 3
  22. 2024A Career Interview Dialogue System using Large Language Model-based Dynamic Slot Generationcited by 3
  23. 2022EVI: Multilingual Spoken Dialogue Tasks and Dataset for Knowledge-Based Enrolment, Verification, and Identificationcited by 2
  24. 2023Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systemscited by 1
  25. 2023HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Fillingcited by 1
  26. 2024On the Use of Audio to Improve Dialogue Policiescited by 1
  27. 2023Multi-View Zero-Shot Open Intent Induction from Dialogues: Multi Domain Batch and Proxy Gradient Transfercited by 0
  28. 2023Actively Discovering New Slots for Task-oriented Conversationcited by 0
  29. 2024Efficient slot labellingcited by 0
  30. 2025Fin-ExBERT: User Intent based Text Extraction in Financial Context using Graph-Augmented BERT and trainable Plugincited by 0