# MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure

arXiv: https://arxiv.org/abs/2210.12487 · HTML: https://mlsanity.com/paper/2210.12487/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 17 of its 43 references that are in the map.
- First posted on arXiv 22 Oct 2022.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 17 of 17)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language](https://mlsanity.com/paper/2012.13048/) (2020)
- [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [Explaining Answers with Entailment Trees](https://mlsanity.com/paper/2104.08661/) (2021)
- [MuTual: A Dataset for Multi-Turn Dialogue Reasoning](https://mlsanity.com/paper/2004.04494/) (2020)
- [Parsing Argumentation Structures in Persuasive Essays](https://mlsanity.com/paper/1604.07370/) (2016)
- [ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning](https://mlsanity.com/paper/2104.07644/) (2021)
- [Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering](https://mlsanity.com/paper/2010.03274/) (2020)
- [Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text](https://mlsanity.com/paper/2105.03659/) (2021)
- [R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason](https://mlsanity.com/paper/1910.04601/) (2019)
- [DAGN: Discourse-Aware Graph Network for Logical Reasoning](https://mlsanity.com/paper/2103.14349/) (2021)
- [METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanation](https://mlsanity.com/paper/2205.02593/) (2022)
- [Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.00731/) (2022)

## Built on by (most cited first, 3 of 3)

- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [Faithful Question Answering with Monte-Carlo Planning](https://mlsanity.com/paper/2305.02556/) (2023)
- [CLOMO: Counterfactual Logical Modification with Large Language Models](https://mlsanity.com/paper/2311.17438/) (2023)

## Abstract (arXiv)

In this paper, we propose a comprehensive benchmark to investigate models' logical reasoning capabilities in complex real-life scenarios. Current explanation datasets often employ synthetic data with simple reasoning structures. Therefore, it cannot express more complex reasoning processes, such as the rebuttal to a reasoning step and the degree of certainty of the evidence. To this end, we propose a comprehensive logical reasoning explanation form. Based on the multi-hop chain of reasoning, the explanation form includes three main components: (1) The condition of rebuttal that the reasoning node can be challenged; (2) Logical formulae that uncover the internal texture of reasoning nodes; (3) Reasoning strength indicated by degrees of certainty. The fine-grained structure conforms to the real logical reasoning scenario, better fitting the human cognitive process but, simultaneously, is more challenging for the current models. We evaluate the current best models' performance on this new explanation form. The experimental results show that generating reasoning graphs remains a challenging task for current models, even with the help of giant pre-trained language models.