mlsanityData to 7 Oct 2026

MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning

arXiv 2203.00357 ↗First posted 1 Mar 2022cs.CL cs.AIcited by 22

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Logical reasoning is of vital importance to natural language understanding. Previous studies either employ graph-based models to incorporate prior knowledge about logical relations, or introduce symbolic logic into neural models through data augmentation. These methods, however, heavily depend on annotated training data, and thus suffer from over-fitting and poor generalization problems due to the dataset sparsity. To address these two problems, in this paper, we propose MERIt, a MEta-path guided contrastive learning method for logical ReasonIng of text, to perform self-supervised pre-training on abundant unlabeled text data. Two novel strategies serve as indispensable components of our method. In particular, a strategy based on meta-path is devised to discover the logical structure in natural texts, followed by a counterfactual data augmentation strategy to eliminate the information shortcut induced by pre-training. The experimental results on two challenging logical reasoning benchmarks, i.e., ReClor and LogiQA, demonstrate that our method outperforms the SOTA baselines with significant improvements.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 35 references that are in the map. First posted on arXiv 1 Mar 2022.

Builds on

28 of 35 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 671
  10. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  11. 2020Transformers as Soft Reasoners over Languagecited by 265
  12. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
16 more
  1. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  2. 2019DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehensioncited by 156
  3. 2020Double Graph Based Reasoning for Document-level Relation Extractioncited by 87
  4. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 83
  5. 2020ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learningcited by 72
  6. 2021Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Textcited by 46
  7. 2021Why Machine Reading Comprehension Models Learn Shortcuts?cited by 40
  8. 2020Loss re-scaling VQA: Revisiting the LanguagePrior Problem from a Class-imbalance Viewcited by 36
  9. 2020Critical Thinking for Language Modelscited by 31
  10. 2021SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extractioncited by 30
  11. 2021DAGN: Discourse-Aware Graph Network for Logical Reasoningcited by 28
  12. 2021Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphscited by 25
  13. 2021Discriminative Reasoning for Document-level Relation Extractioncited by 25
  14. 2021ReasonBERT: Pre-trained to Reason with Distant Supervisioncited by 22
  15. 2019Quantifying and Alleviating the Language Prior Problem in Visual Question Answeringcited by 13
  16. 2021REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-trainingcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 13; 2024: 2; 2025: 3.

22 papers.

Sort:

By area:

  1. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 199
  2. 2023SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoningcited by 94
  3. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 94
  4. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyondcited by 65
  5. 2023Natural Language Reasoning, A Surveycited by 61
  6. 2023Large Language Models Are Not Strong Abstract Reasonerscited by 52
  7. 2022Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structurecited by 43
  8. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
  9. 2023DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacycited by 34
  10. 2025VChain: Chain-of-Visual-Thought for Reasoning in Video Generationcited by 31
  11. 2024PUZZLES: A Benchmark for Neural Algorithmic Reasoningcited by 12
  12. 2023Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoningcited by 10
10 more
  1. 2023Exploring Self-supervised Logic-enhanced Training for Large Language Modelscited by 8
  2. 2023CLOMO: Counterfactual Logical Modification with Large Language Modelscited by 7
  3. 2022APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoningcited by 5
  4. 2023IDOL: Indicator-oriented Logic Pre-training for Logical Reasoningcited by 3
  5. 2022Discourse-Aware Graph Networks for Textual Logical Reasoningcited by 2
  6. 2023Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Textcited by 2
  7. 2023Deep Manifold Learning for Reading Comprehension and Logical Reasoning Tasks with Polytuplet Losscited by 0
  8. 2023Proposition from the Perspective of Chinese Language: A Chinese Proposition Classification Evaluation Benchmarkcited by 0
  9. 2025Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentationcited by 0
  10. 2025LFC-DA: Logical Formula-Controlled Data Augmentation for Enhanced Logical Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension
  2. 2025Teaching Small Language Models to Learn Logic through Meta-Learning
  3. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning
  4. 2025Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation
  5. 2026CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning
  6. 2026SymbolLKG: Towards Verifiable Logical Reasoning via Logical Knowledge Graph and Symbolic Solvers
  7. 2025Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions
  8. 2025Hybrid Models for Natural Language Reasoning: The Case of Syllogistic Logic