# Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset

arXiv: https://arxiv.org/abs/2311.05113 · HTML: https://mlsanity.com/paper/2311.05113/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 9 Nov 2023.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 10 of 10)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Evaluating the Performance of Large Language Models on GAOKAO Benchmark](https://mlsanity.com/paper/2305.12474/) (2023)
- [A Survey of Deep Learning for Mathematical Reasoning](https://mlsanity.com/paper/2212.10535/) (2022)
- [Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems](https://mlsanity.com/paper/2009.11506/) (2020)
- [Crosslingual Generalization through Multitask Finetuning](https://mlsanity.com/paper/2211.01786/) (2022)

## Built on by (most cited first, 5 of 5)

- [Cost-Performance Optimization for Processing Low-Resource Language Tasks Using Commercial LLMs](https://mlsanity.com/paper/2403.05434/) (2024)
- [A Survey of Deep Learning for Geometry Problem Solving](https://mlsanity.com/paper/2507.11936/) (2025)
- [From Recall to Reasoning: Automated Question Generation for Deeper Math Learning through Large Language Models](https://mlsanity.com/paper/2505.11899/) (2025)
- [FormalAnalyticGeo: A Neural-Symbolic Based Framework for Multimodal Analytic Geometry Problem Generation](https://mlsanity.com/paper/2607.12982/) (2026)
- [Dense Neuro-Symbolic Reasoning in a Unified Geometry State](https://mlsanity.com/paper/2610.04280/) (2026)

## Abstract (arXiv)

Mathematical understanding and reasoning are crucial tasks for assessing the capabilities of artificial intelligence (AI). However, existing benchmarks either require just a few steps of reasoning, or only contain a small amount of data in one specific topic, making it hard to analyse AI's behaviour with reference to different problems within a specific topic in detail. In this work, we propose Conic10K, a challenging math problem dataset on conic sections in Chinese senior high school education. Our dataset contains various problems with different reasoning depths, while only the knowledge from conic sections is required. Since the dataset only involves a narrow range of knowledge, it is easy to separately analyse the knowledge a model possesses and the reasoning ability it has. For each problem, we provide a high-quality formal representation, the reasoning steps, and the final solution. Experiments show that existing large language models, including GPT-4, exhibit weak performance on complex reasoning. We hope that our findings could inspire more advanced techniques for precise natural language understanding and reasoning. Our dataset and codes are available at https://github.com/whyNLP/Conic10K.