mlsanityData to 7 Oct 2026

Logically Consistent Adversarial Attacks for Soft Theorem Provers

arXiv 2205.00047 ↗First posted 29 Apr 2022cs.LG cs.CL cs.CRcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent efforts within the AI community have yielded impressive results towards "soft theorem proving" over natural language sentences using language models. We propose a novel, generative adversarial framework for probing and improving these models' reasoning capabilities. Adversarial attacks in this domain suffer from the logical inconsistency problem, whereby perturbations to the input may alter the label. Our Logically consistent AdVersarial Attacker, LAVA, addresses this by combining a structured generative process with a symbolic solver, guaranteeing logical consistency. Our framework successfully generates adversarial attacks and identifies global weaknesses common across multiple target models. Our analyses reveal naive heuristics and vulnerabilities in these models' reasoning capabilities, exposing an incomplete grasp of logical deduction under logic programs. Finally, in addition to effective probing of these models, we show that training on the generated samples improves the target model's performance.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 29 Apr 2022.

Builds on

23 of 33 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  8. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  9. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
  10. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  11. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  12. 2020Transformers as Soft Reasoners over Languagecited by 265
11 more
  1. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  2. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  3. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  4. 2014Neural Variational Inference and Learning in Belief Networkscited by 116
  5. 2020OpenAttack: An Open-source Textual Adversarial Attack Toolkitcited by 84
  6. 2020PRover: Proof Generation for Interpretable Reasoning over Rulescited by 53
  7. 2020Measuring Systematic Generalization in Neural Proof Generation with Transformerscited by 43
  8. 2020Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Primingcited by 41
  9. 2020Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledgecited by 24
  10. 2020Generating Natural Language Adversarial Examples on a Large Scale with Generative Modelscited by 15
  11. 2020Generating Semantically Valid Adversarial Questions for TableQAcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2.

2 papers.

  1. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 17
  2. 2023Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models
  2. 2025Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning
  3. 2025Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions
  4. 2026LLMEval-Logic: A Solver-Verified Chinese Benchmark for Logical Reasoning of LLMs with Adversarial Hardening
  5. 2022LogiGAN: Learning Logical Reasoning via Adversarial Pre-training
  6. 2025Investigating the Robustness of Deductive Reasoning with Large Language Models
  7. 2026Mitigating Adaptive Attacks against Reasoning Models with Activation Consistency Training
  8. 2024Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference