mlsanityData to 7 Oct 2026

On Robustness of Neural Semantic Parsers

arXiv 2102.01563 ↗First posted 2 Feb 2021cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Semantic parsing maps natural language (NL) utterances into logical forms (LFs), which underpins many advanced NLP problems. Semantic parsers gain performance boosts with deep neural networks, but inherit vulnerabilities against adversarial examples. In this paper, we provide the empirical study on the robustness of semantic parsers in the presence of adversarial attacks. Formally, adversaries of semantic parsing are considered to be the perturbed utterance-LF pairs, whose utterances have exactly the same meanings as the original ones. A scalable methodology is proposed to construct robustness test sets based on existing benchmark corpora. Our results answered five research questions in measuring the sate-of-the-art parsers' performance on robustness test sets, and evaluating the effect of data augmentation.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 2 Feb 2021.

Builds on

33 of 38 references.

References · 33

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  6. 2018Robustness May Be at Odds with Accuracycited by 873
  7. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  8. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  9. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  11. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  12. 2019Simple Black-box Adversarial Attackscited by 249
21 more
  1. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  2. 2016Language to Logical Form with Neural Attentioncited by 158
  3. 2016Data Recombination for Neural Semantic Parsingcited by 111
  4. 2018Coarse-to-Fine Decoding for Neural Semantic Parsingcited by 110
  5. 2016Crafting Adversarial Input Sequences for Recurrent Neural Networkscited by 102
  6. 2018Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examplescited by 93
  7. 2018TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generationcited by 88
  8. 2018On Adversarial Examples for Character-Level Neural Machine Translationcited by 85
  9. 2018A Survey on Semantic Parsingcited by 62
  10. 2019On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Modelscited by 56
  11. 2020Neural Syntactic Preordering for Controlled Paraphrase Generationcited by 56
  12. 2019Corpora Generation for Grammatical Error Correctioncited by 53
  13. 2018On the Robustness of the CVPR 2018 White-Box Adversarial Example Defensescited by 40
  14. 2018Natural Language to Structured Query Generation via Meta-Learningcited by 33
  15. 2018Question Generation from SQL Queries Improves Neural Semantic Parsingcited by 29
  16. 2018Sequence-to-Action: End-to-End Semantic Graph Generation for Semantic Parsingcited by 21
  17. 2018Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehensioncited by 19
  18. 2017HotFlip: White-Box Adversarial Examples for Text Classificationcited by 12
  19. 2019Towards a Robust Deep Neural Network in Texts: A Surveycited by 7
  20. 2020Elephant in the Room: An Evaluation Framework for Assessing Adversarial Examples in NLPcited by 7
  21. 2018Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computercited by 3

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 9; 2024: 2.

14 papers.

Sort:

  1. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  2. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 50
  3. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  4. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  5. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  6. 2024Prompt Perturbation Consistency Learning for Robust Language Modelscited by 26
  7. 2023Energy-Latency Attacks to On-Device Neural Networks via Sponge Poisoningcited by 10
  8. 2024Improving Generalization in Semantic Parsing by Increasing Natural Language Variationcited by 4
  9. 2023The Best of Both Worlds: Combining Human and Machine Translations for Multilingual Semantic Parsing with Active Learningcited by 3
  10. 2023Enhancing Robustness of AI Offensive Code Generators via Data Augmentationcited by 3
  11. 2022Can NMT Understand Me? Towards Perturbation-based Evaluation of NMT Models for Code Generationcited by 2
  12. 2023More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Rankingcited by 2
2 more
  1. 2023Beyond the Model: Data Pre-processing Attack to Deep Learning Models in Android Appscited by 2
  2. 2023Grounding Description-Driven Dialogue State Trackers with Knowledge-Seeking Turnscited by 1