mlsanityData to 7 Oct 2026

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

arXiv 2004.10157 ↗First posted 21 Apr 2020cs.CLcited by 77

Not on the map. Why?

Abstract (arXiv)

Many natural language questions require qualitative, quantitative or logical comparisons between two entities or events. This paper addresses the problem of improving the accuracy and consistency of responses to comparison questions by integrating logic rules and neural models. Our method leverages logical and linguistic knowledge to augment labeled training data and then uses a consistency-based regularizer to train the model. Improving the global consistency of predictions, our approach achieves large improvements over previous methods in a variety of question answering (QA) tasks including multiple-choice qualitative reasoning, cause-effect reasoning, and extractive machine reading comprehension. In particular, our method significantly improves the performance of RoBERTa-based models by 1-5% across datasets. We advance the state of the art by around 5-8% on WIQA and QuaRel and reduce consistency violations by 58% on HotpotQA. We further demonstrate that our approach can learn effectively from limited data.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 12; 2023: 16; 2024: 11; 2025: 11; 2026: 1.

77 papers.

Sort:

By area:

  1. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  2. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  3. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  4. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  5. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  6. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  7. 2021Data Augmentation Approaches in Natural Language Processing: A Surveycited by 82
  8. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  9. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  10. 2025Empowering LLMs with Logical Reasoning: A Comprehensive Surveycited by 51
  11. 2022Reasoning Like Program Executorscited by 45
  12. 2022Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inferencecited by 40
38 more
  1. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  2. 2023Consistency Analysis of ChatGPTcited by 37
  3. 2025R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learningcited by 32
  4. 2023Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChaincited by 31
  5. 2023The Trickle-down Impact of Reward (In-)consistency on RLHFcited by 29
  6. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  7. 2021Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answeringcited by 27
  8. 2021"I'm Not Mad": Commonsense Implications of Negation and Contradictioncited by 26
  9. 2021Substructure Substitution: Structured Data Augmentation for NLPcited by 25
  10. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23
  11. 2024A Survey on Natural Language Counterfactual Generationcited by 21
  12. 2021Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understandingcited by 19
  13. 2024TrustScore: Reference-Free Evaluation of LLM Response Trustworthinesscited by 19
  14. 2021Exploring Transitivity in Neural NLI Models through Veridicalitycited by 15
  15. 2021Teaching Autoregressive Language Models Complex Tasks By Demonstrationcited by 15
  16. 2023Separating form and meaning: Using self-consistency to quantify task understanding across multiple sensescited by 15
  17. 2024Logical Consistency of Large Language Models in Fact-checkingcited by 15
  18. 2025Consistency in Language Models: Current Landscape, Challenges, and Future Directionscited by 14
  19. 2021Accurate, yet inconsistent? Consistency Analysis on Language Understanding Modelscited by 13
  20. 2024One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generationscited by 13
  21. 2021DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learningcited by 12
  22. 2023Semantic Strengthening of Neuro-Symbolic Learningcited by 12
  23. 2021Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decompositioncited by 11
  24. 2022Toward Learning Robust and Invariant Representations with Alignment Regularization and Data Augmentationcited by 11
  25. 2025Controllable Logical Hypothesis Generation for Abductive Reasoning in Knowledge Graphscited by 11
  26. 2025Formalizing Learning from Language Feedback with Provable Guaranteescited by 10
  27. 2021Consistency Training with Virtual Adversarial Discrete Perturbationcited by 9
  28. 2023GLUECons: A Generic Benchmark for Learning Under Constraintscited by 9
  29. 2021Learning with Instance Bundles for Reading Comprehensioncited by 8
  30. 2021Evaluating Relaxations of Logic for Neural Networks: A Comprehensive Studycited by 8
  31. 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarkscited by 8
  32. 2023Teaching Probabilistic Logical Reasoning to Transformerscited by 7
  33. 2023Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Modelscited by 7
  34. 2024From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistencycited by 7
  35. 2021Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Datacited by 6
  36. 2021Paired Examples as Indirect Supervision in Latent Decision Modelscited by 6
  37. 2025Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptionscited by 6
  38. 2021Beyond the Tip of the Iceberg: Assessing Coherence of Text Classifierscited by 5

50 most cited of 77 All 77 (JSON)