mlsanityData to 7 Oct 2026

SkyMath: Technical Report

arXiv 2310.16713 ↗First posted 25 Oct 2023cs.CL cs.AIcited by 1

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large language models (LLMs) have shown great potential to solve varieties of natural language processing (NLP) tasks, including mathematical reasoning. In this work, we present SkyMath, a large language model for mathematics with 13 billion parameters. By applying self-compare fine-tuning, we have enhanced mathematical reasoning abilities of Skywork-13B-Base remarkably. On GSM8K, SkyMath outperforms all known open-source models of similar size and has established a new SOTA performance.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 53 of its 62 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

53 of 62 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text10 Oct
  2. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct18 Aug

Other references · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  10. 2023Visual Instruction Tuningcited by 7,792
  11. 2020Scaling Laws for Neural Language Modelscited by 6,411
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
38 more
  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  11. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  13. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  14. 2023BloombergGPT: A Large Language Model for Financecited by 923
  15. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  16. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  17. 2023StarCoder: may the source be with you!cited by 790
  18. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  19. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  20. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  21. 2022Large Language Models Can Self-Improvecited by 656
  22. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  23. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  24. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  25. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  26. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  27. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  28. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  29. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244
  30. 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 161
  31. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  32. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 119
  33. 2023CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?cited by 89
  34. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  35. 2023Instruction Mining: Instruction Data Selection for Tuning Large Language Modelscited by 57
  36. 2023LLaSM: Large Language and Speech Modelcited by 56
  37. 2023Radiology-GPT: A Large Language Model for Radiologycited by 44
  38. 2023LLark: A Multimodal Instruction-Following Language Model for Musiccited by 29

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18

Similar, not cited

Close in wording, no citation either way

  1. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
  2. 2024ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models
  3. 2024LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning
  4. 2024FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
  5. 2024MathHay: An Automated Benchmark for Long-Context Mathematical Reasoning in LLMs
  6. 2025Can LLMs textitunderstand Math? -- Exploring the Pitfalls in Mathematical Reasoning
  7. 2024Logic Contrastive Reasoning with Lightweight Large Language Model for Math Word Problems
  8. 2025Evaluation of LLMs for mathematical problem solving