mlsanityData to 7 Oct 2026

PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts

arXiv 2306.04535 ↗First posted 7 Jun 2023cs.CLcited by 6

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

A key component of modern conversational systems is the Dialogue State Tracker (or DST), which models a user's goals and needs. Toward building more robust and reliable DSTs, we introduce a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models. Two key characteristics of this approach are: (i) it only needs the output of the DST with no need for model parameters, and (ii) it can learn to generate natural language utterances that can target any DST. Through experiments over state-of-the-art DSTs, the proposed framework leads to the greatest reduction in accuracy and the best attack success rate while maintaining good fluency and a low perturbation ratio. We also show how much the generated adversarial examples can bolster a DST through adversarial training. These results indicate the strength of prompt-based attacks on DSTs and leave open avenues for continued refinement.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 35 of its 46 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

35 of 46 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  9. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  10. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  11. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  12. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
23 more
  1. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  2. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  3. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  4. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  5. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  6. 2020TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Trackingcited by 130
  7. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  8. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  9. 2020DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialoguecited by 89
  10. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  11. 2022HyperPrompt: Prompt-based Task-Conditioning of Transformerscited by 67
  12. 2021Few-Shot Bot: Prompt-Based Learning for Dialogue Systemscited by 64
  13. 2022Continual Prompt Tuning for Dialog State Trackingcited by 44
  14. 2020CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackerscited by 43
  15. 2020A Sequence-to-Sequence Approach to Dialogue State Trackingcited by 43
  16. 2021Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Trackingcited by 35
  17. 2020Robustness Testing of Language Understanding in Task-Oriented Dialogcited by 32
  18. 2022Phrase-level Textual Adversarial Attack with Label Preservationcited by 23
  19. 2021Effective Sequence-to-Sequence Dialogue State Trackingcited by 20
  20. 2022A Prompting-based Approach for Adversarial Example Generation and Robustness Enhancementcited by 12
  21. 2023HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Promptingcited by 11
  22. 2022A Dual Prompt Learning Framework for Few-Shot Dialogue State Trackingcited by 8
  23. 2022Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typingcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 2.

6 papers.

  1. 2023Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuningcited by 15
  2. 2023A Systematic Survey of Prompt Engineering on Vision-Language Foundation Modelscited by 12
  3. 2023TrojFSP: Trojan Insertion in Few-shot Prompt Tuningcited by 9
  4. 2024Enhance Modality Robustness in Text-Centric Multimodal Alignment with Adversarial Promptingcited by 5
  5. 2024Prompt-based Unifying Inference Attack on Graph Neural Networkscited by 3
  6. 2023DA3: A Distribution-Aware Adversarial Attack against Language Modelscited by 1