mlsanityData to 14 Oct 2026

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling

arXiv 2208.11508 ↗First posted 24 Aug 2022cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. However, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed semantic structure awareness transferring method for training perturbation-robust slot filling models. Specifically, we introduce two MLM-based training strategies to respectively learn contextual semantic structure and word distribution from unsupervised language perturbation corpus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aim to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 50 of its 84 references that are in the map. First posted on arXiv 24 Aug 2022.

Builds on

50 of 84 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  6. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  7. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  8. 2016Neural Architectures for Named Entity Recognitioncited by 550
  9. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 377
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  11. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  12. 2015Named Entity Recognition with Bidirectional LSTM-CNNscited by 201
38 more
  1. 2016Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Fillingcited by 137
  2. 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 111
  3. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  4. 2019A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Fillingcited by 85
  5. 2020A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generationcited by 72
  6. 2018Sequence-to-Sequence Data Augmentation for Dialogue Language Understandingcited by 68
  7. 2018Text Data Augmentation Made Simple By Leveraging NLP Cloud APIscited by 40
  8. 2020RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systemscited by 36
  9. 2016Adversarial Diversity and Hard Positive Generationcited by 35
  10. 2020Rethinking Generalization of Neural Models: A Named Entity Recognition Case Studycited by 35
  11. 2020Robustness Testing of Language Understanding in Task-Oriented Dialogcited by 32
  12. 2022InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NERcited by 31
  13. 2020Joint Contextual Modeling for ASR Correction and Language Understandingcited by 30
  14. 2021Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrificecited by 30
  15. 2021RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Modelscited by 28
  16. 2020Conditional Augmentation for Aspect Term Extraction via Masked Sequence-to-Sequence Generationcited by 27
  17. 2017How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?cited by 23
  18. 2021Phoneme-BERT: Joint Language Modelling of Phoneme Sequence and ASR Transcriptcited by 21
  19. 2019Learning ASR-Robust Contextualized Embeddings for Spoken Language Understandingcited by 20
  20. 2016Joint Online Spoken Language Understanding and Language Modeling with Recurrent Neural Networkscited by 19
  21. 2020Are Neural Open-Domain Dialog Systems Robust to Speech Recognition Errors in the Dialog History? An Empirical Studycited by 19
  22. 2020NAT: Noise-Aware Training for Robust Neural Sequence Labelingcited by 17
  23. 2022MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspectivecited by 17
  24. 2020Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improvecited by 16
  25. 2021Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Fillingcited by 13
  26. 2022Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Taskscited by 13
  27. 2022Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defensecited by 13
  28. 2020Improving Spoken Language Understanding By Exploiting ASR N-best Hypothesescited by 12
  29. 2022A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancementcited by 12
  30. 2020C2C-GenDA: Cluster-to-Cluster Generation for Data Augmentation of Slot Fillingcited by 11
  31. 2022ASSIST: Towards Label Noise-Robust Dialogue State Trackingcited by 11
  32. 2020Data Augmentation for Spoken Language Understanding via Pretrained Language Modelscited by 9
  33. 2021Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understandingcited by 9
  34. 2021ASR-GLUE: A New Multi-task Benchmark for ASR-Robust Natural Language Understandingcited by 9
  35. 2021Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labelingcited by 8
  36. 2020A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land?cited by 6
  37. 2020Noise Robust Named Entity Understanding for Voice Assistantscited by 5
  38. 2021Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understandingcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 3.

9 papers.

  1. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  2. 2023Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Taskcited by 23
  3. 2023Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQAcited by 17
  4. 2023Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPTcited by 16
  5. 2023A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NERcited by 9
  6. 2023Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbationscited by 5
  7. 2023Watch the Speakers: A Hybrid Continuous Attribution Network for Emotion Recognition in Conversation With Emotion Disentanglementcited by 1
  8. 2024Noise-BERT: A Unified Perturbation-Robust Framework with Noise Alignment Pre-training for Noisy Slot Filling Taskcited by 1
  9. 2024Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learningcited by 0