mlsanityData to 14 Oct 2026

Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbations

arXiv 2310.03518 ↗First posted 5 Oct 2023cs.CL cs.AI cs.DScited by 5

Not on the map. Why?

Abstract (arXiv)

In real dialogue scenarios, as there are unknown input noises in the utterances, existing supervised slot filling models often perform poorly in practical applications. Even though there are some studies on noise-robust models, these works are only evaluated on rule-based synthetic datasets, which is limiting, making it difficult to promote the research of noise-robust methods. In this paper, we introduce a noise robustness evaluation dataset named Noise-SF for slot filling task. The proposed dataset contains five types of human-annotated noise, and all those noises are exactly existed in real extensive robust-training methods of slot filling into the proposed framework. By conducting exhaustive empirical evaluation experiments on Noise-SF, we find that baseline models have poor performance in robustness evaluation, and the proposed framework can effectively improve the robustness of models. Based on the empirical experimental results, we make some forward-looking suggestions to fuel the research in this direction. Our dataset Noise-SF will be released at https://github.com/dongguanting/Noise-SF.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

39 of 59 references.

References · 39

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  4. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  5. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  6. 2016Neural Architectures for Named Entity Recognitioncited by 550
  7. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  8. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  9. 2018Adversarial Attacks and Defences: A Surveycited by 259
  10. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  11. 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
  12. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
27 more
  1. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  2. 2020A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generationcited by 72
  3. 2018Text Data Augmentation Made Simple By Leveraging NLP Cloud APIscited by 40
  4. 2020RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systemscited by 36
  5. 2016Adversarial Diversity and Hard Positive Generationcited by 35
  6. 2020Rethinking Generalization of Neural Models: A Named Entity Recognition Case Studycited by 35
  7. 2020Robustness Testing of Language Understanding in Task-Oriented Dialogcited by 32
  8. 2017How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?cited by 23
  9. 2019Learning ASR-Robust Contextualized Embeddings for Spoken Language Understandingcited by 20
  10. 2016Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networkscited by 19
  11. 2020Are Neural Open-Domain Dialog Systems Robust to Speech Recognition Errors in the Dialog History? An Empirical Studycited by 19
  12. 2020NAT: Noise-Aware Training for Robust Neural Sequence Labelingcited by 17
  13. 2023Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQAcited by 17
  14. 2020Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improvecited by 16
  15. 2022Semi-Supervised Knowledge-Grounded Pre-training for Task-Oriented Dialog Systemscited by 15
  16. 2021Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Fillingcited by 13
  17. 2020Improving Spoken Language Understanding By Exploiting ASR N-best Hypothesescited by 12
  18. 2020Data Augmentation for Spoken Language Understanding via Pretrained Language Modelscited by 9
  19. 2021Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understandingcited by 9
  20. 2022PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Fillingcited by 9
  21. 2023A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NERcited by 9
  22. 2021Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labelingcited by 8
  23. 2023Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Frameword with Multi-level Data Augmentationscited by 8
  24. 2023Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Promptingcited by 8
  25. 2022A Robust Contrastive Alignment Method For Multi-Domain Text Classificationcited by 6
  26. 2023A Prototypical Semantic Decoupling Method via Joint Contrastive Learning for Few-Shot Name Entity Recognitioncited by 6
  27. 2023Watch the Speakers: A Hybrid Continuous Attribution Network for Emotion Recognition in Conversation With Emotion Disentanglementcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 1.

5 papers.

  1. 2023Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Taskcited by 23
  2. 2023Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPTcited by 16
  3. 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Setscited by 6
  4. 2023DemoNSF: A Multi-task Demonstration-based Generative Framework for Noisy Slot Filling Taskcited by 5
  5. 2024Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Taskcited by 1