mlsanityData to 7 Oct 2026

SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinese

arXiv 2401.11819 ↗First posted 22 Jan 2024cs.CL cs.AIcited by 10

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

We introduce SuperCLUE-Math6(SC-Math6), a new benchmark dataset to evaluate the mathematical reasoning abilities of Chinese language models. SC-Math6 is designed as an upgraded Chinese version of the GSM8K dataset with enhanced difficulty, diversity, and application scope. It consists of over 2000 mathematical word problems requiring multi-step reasoning and providing natural language solutions. We propose an innovative scheme to quantify the reasoning capability of large models based on performance over problems with different reasoning steps. Experiments on 13 representative Chinese models demonstrate a clear stratification of reasoning levels, with top models like GPT-4 showing superior performance. SC-Math6 fills the gap in Chinese mathematical reasoning benchmarks and provides a comprehensive testbed to advance the intelligence of Chinese language models.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 7 references that are in the map. First posted on arXiv 22 Jan 2024.

Builds on

6 of 7 references.

References · 6

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 194

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 1.

10 papers.

  1. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  2. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  3. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contextscited by 32
  4. 2025Benchmarking Reasoning Robustness in Large Language Modelscited by 30
  5. 2024Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones?cited by 9
  6. 2024BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Competitive Large Language Model Baselinecited by 6
  7. 2025MathDebugger: Detecting and Diagnosing Errors in Synthetic Mathematical Datacited by 6
  8. 2024Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrievalcited by 4
  9. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Modelscited by 2
  10. 2026From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Benchcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
  2. 2024ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models
  3. 2025MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning
  4. 2025Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
  5. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models
  6. 2025UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models
  7. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  8. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models