mlsanityData to 7 Oct 2026

Com2: A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models

arXiv 2506.07064 ↗First posted 8 Jun 2025cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) have mastered abundant simple and explicit commonsense knowledge through pre-training, enabling them to achieve human-like performance in simple commonsense reasoning. Nevertheless, LLMs struggle to reason with complex and implicit commonsense knowledge that is derived from simple ones (such as understanding the long-term effects of certain events), an aspect humans tend to focus on more. Existing works focus on complex tasks like math and code, while complex commonsense reasoning remains underexplored due to its uncertainty and lack of structure. To fill this gap and align with real-world concerns, we propose a benchmark Com2 focusing on complex commonsense reasoning. We first incorporate causal event graphs to serve as structured complex commonsense. Then we adopt causal theory~(e.g., intervention) to modify the causal event graphs and obtain different scenarios that meet human concerns. Finally, an LLM is employed to synthesize examples with slow thinking, which is guided by the logical relationships in the modified causal graphs. Furthermore, we use detective stories to construct a more challenging subset. Experiments show that LLMs struggle in reasoning depth and breadth, while post-training and slow thinking can alleviate this. The code and data are available at https://github.com/Waste-Wood/Com2.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 15 of its 42 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

15 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024DeepSeek-V3 Technical Report27 Dec
  2. 2024Multi-Step Reasoning with Large Language Models, a Survey16 Jul

Other references · 13

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  6. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  7. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  8. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 142
  9. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 109
  10. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Modelcited by 62
  11. 2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
  12. 2019ELG: An Event Logic Graphcited by 8
1 more
  1. 2024Diagnosing and Remedying Knowledge Deficiencies in LLMs via Label-free Curricular Meaningful Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs
  2. 2024CARL-GT: Evaluating Causal Reasoning Capabilities of Large Language Models
  3. 2024A Critical Review of Causal Reasoning Benchmarks for Large Language Models
  4. 2024COLD: Causal reasOning in cLosed Daily activities
  5. 2023CLadder: Assessing Causal Reasoning in Language Models
  6. 2022Language Models of Code are Few-Shot Commonsense Learners
  7. 2025Causal Distillation: Transferring Structured Explanations from Large to Compact Language Models
  8. 2026CommonWhy: A Dataset for Evaluating Entity-Based Causal Commonsense Reasoning in Large Language Models