# MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning

arXiv: https://arxiv.org/abs/2203.00357 · HTML: https://mlsanity.com/paper/2203.00357/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 1 Mar 2022.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Large Batch Optimization for Deep Learning: Training BERT in 76 minutes](https://mlsanity.com/paper/1904.00962/) (2019)
- [GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training](https://mlsanity.com/paper/2006.09963/) (2020)
- [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension](https://mlsanity.com/paper/1902.00164/) (2019)
- [Double Graph Based Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2009.13752/) (2020)
- [Look at the First Sentence: Position Bias in Question Answering](https://mlsanity.com/paper/2004.14602/) (2020)
- [ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning](https://mlsanity.com/paper/2012.15022/) (2020)
- [Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text](https://mlsanity.com/paper/2105.03659/) (2021)
- [Why Machine Reading Comprehension Models Learn Shortcuts?](https://mlsanity.com/paper/2106.01024/) (2021)
- [Loss re-scaling VQA: Revisiting the LanguagePrior Problem from a Class-imbalance View](https://mlsanity.com/paper/2010.16010/) (2020)
- [Critical Thinking for Language Models](https://mlsanity.com/paper/2009.07185/) (2020)
- [SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01709/) (2021)
- [DAGN: Discourse-Aware Graph Network for Logical Reasoning](https://mlsanity.com/paper/2103.14349/) (2021)
- [Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2103.10367/) (2021)
- [Discriminative Reasoning for Document-level Relation Extraction](https://mlsanity.com/paper/2106.01562/) (2021)
- [ReasonBERT: Pre-trained to Reason with Distant Supervision](https://mlsanity.com/paper/2109.04912/) (2021)
- [Quantifying and Alleviating the Language Prior Problem in Visual Question Answering](https://mlsanity.com/paper/1905.04877/) (2019)
- [REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training](https://mlsanity.com/paper/2105.04201/) (2021)

## Built on by (most cited first, 22 of 22)

- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2406.09136/) (2024)
- [SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning](https://mlsanity.com/paper/2309.04766/) (2023)
- [Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond](https://mlsanity.com/paper/2306.09841/) (2023)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Large Language Models Are Not Strong Abstract Reasoners](https://mlsanity.com/paper/2305.19555/) (2023)
- [Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure](https://mlsanity.com/paper/2211.10738/) (2022)
- [Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models](https://mlsanity.com/paper/2310.00836/) (2023)
- [DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy](https://mlsanity.com/paper/2310.18659/) (2023)
- [VChain: Chain-of-Visual-Thought for Reasoning in Video Generation](https://mlsanity.com/paper/2510.05094/) (2025)
- [PUZZLES: A Benchmark for Neural Algorithmic Reasoning](https://mlsanity.com/paper/2407.00401/) (2024)
- [Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoning](https://mlsanity.com/paper/2310.09430/) (2023)
- [Exploring Self-supervised Logic-enhanced Training for Large Language Models](https://mlsanity.com/paper/2305.13718/) (2023)
- [CLOMO: Counterfactual Logical Modification with Large Language Models](https://mlsanity.com/paper/2311.17438/) (2023)
- [APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning](https://mlsanity.com/paper/2212.09282/) (2022)
- [IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning](https://mlsanity.com/paper/2306.15273/) (2023)
- [Discourse-Aware Graph Networks for Textual Logical Reasoning](https://mlsanity.com/paper/2207.01450/) (2022)
- [Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Text](https://mlsanity.com/paper/2301.02983/) (2023)
- [Deep Manifold Learning for Reading Comprehension and Logical Reasoning Tasks with Polytuplet Loss](https://mlsanity.com/paper/2304.01046/) (2023)
- [Proposition from the Perspective of Chinese Language: A Chinese Proposition Classification Evaluation Benchmark](https://mlsanity.com/paper/2309.09602/) (2023)
- [Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation](https://mlsanity.com/paper/2502.19907/) (2025)
- [LFC-DA: Logical Formula-Controlled Data Augmentation for Enhanced Logical Reasoning](https://mlsanity.com/paper/2511.03372/) (2025)

## Abstract (arXiv)

Logical reasoning is of vital importance to natural language understanding. Previous studies either employ graph-based models to incorporate prior knowledge about logical relations, or introduce symbolic logic into neural models through data augmentation. These methods, however, heavily depend on annotated training data, and thus suffer from over-fitting and poor generalization problems due to the dataset sparsity. To address these two problems, in this paper, we propose MERIt, a MEta-path guided contrastive learning method for logical ReasonIng of text, to perform self-supervised pre-training on abundant unlabeled text data. Two novel strategies serve as indispensable components of our method. In particular, a strategy based on meta-path is devised to discover the logical structure in natural texts, followed by a counterfactual data augmentation strategy to eliminate the information shortcut induced by pre-training. The experimental results on two challenging logical reasoning benchmarks, i.e., ReClor and LogiQA, demonstrate that our method outperforms the SOTA baselines with significant improvements.