mlsanityData to 14 Oct 2026

The Karp Dataset

arXiv 2501.14705 ↗First posted 24 Jan 2025cs.LG cs.CLcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Understanding the mathematical reasoning capabilities of Large Language Models (LLMs) is a central topic in the study of artificial intelligence. This new domain necessitates the creation of datasets of reasoning tasks for both training and benchmarking the performance of LLMs. To this end, we introduce the Karp dataset: The first dataset composed of detailed proofs of NP-completeness reductions. The reductions vary in difficulty, ranging from simple exercises of undergraduate courses to more challenging reductions from academic papers. We compare the performance of state-of-the-art models on this task and demonstrate the effect of fine-tuning with the Karp dataset on reasoning capacity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 31 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

14 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Llemma: An Open Language Model For Mathematics16 Oct
  2. 2022Language Models are Multilingual Chain-of-Thought Reasoners6 Oct

Other references · 12

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  8. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  9. 2022HyperTree Proof Search for Neural Theorem Provingcited by 197
  10. 2022Formal Mathematics Statement Curriculum Learningcited by 142
  11. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 106
  12. 2013On the parameterized complexity of cutting a few vertices from a graphcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
  2. 2025ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models
  3. 2026Improving Mathematical Reasoning Capabilities in Large Language Models via Reasoning Process Error Classification
  4. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
  5. 2026How Well Do LLMs Perform on the Simplest Long-Chain Reasoning Tasks: An Empirical Study on the Equivalence Class Problem
  6. 2025JT-Math: A Multi-Stage Framework for Advanced Mathematical Reasoning in Large Language Models
  7. 2025Large Language Models and Mathematical Reasoning Failures
  8. 2024CLR-Bench: Evaluating Large Language Models in College-level Reasoning