mlsanityData to 7 Oct 2026

KwaiYiiMath: Technical Report

arXiv 2310.07488 ↗First posted 11 Oct 2023cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning. In this report, we introduce the KwaiYiiMath which enhances the mathematical reasoning abilities of KwaiYiiBase1, by applying Supervised Fine-Tuning (SFT) and Reinforced Learning from Human Feedback (RLHF), including on both English and Chinese mathematical tasks. Meanwhile, we also constructed a small-scale Chinese primary school mathematics test set (named KMath), consisting of 188 examples to evaluate the correctness of the problem-solving process generated by the models. Empirical studies demonstrate that KwaiYiiMath can achieve state-of-the-art (SOTA) performance on GSM8k, CMath, and KMath compared with the similar size models, respectively.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 106 of its 123 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

106 of 123 references.

References · 106

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  10. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  11. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  12. 2023Visual Instruction Tuningcited by 7,792
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  6. 2023Qwen Technical Reportcited by 3,273
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  13. 2023A Survey of Large Language Modelscited by 2,537
  14. 2022Emergent Abilities of Large Language Modelscited by 2,483
  15. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  16. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  17. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  18. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  19. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  20. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  21. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  22. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  23. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  24. 2023PaLM 2 Technical Reportcited by 1,154
  25. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  26. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  27. 2023LIMA: Less Is More for Alignmentcited by 1,080
  28. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  29. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  30. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  31. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  32. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  33. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  34. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  35. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  36. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  37. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  38. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712

50 most cited of 106 All 106 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarioscited by 18
  2. 2024Embedding Self-Correction as an Inherent Ability in Large Language Models for Enhanced Mathematical Reasoningcited by 13
  3. 2025Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stagescited by 13
  4. 2023SkyMath: Technical Reportcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models
  2. 2024LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning
  3. 2025A Survey on Large Language Models for Mathematical Reasoning
  4. 2025JT-Math: A Multi-Stage Framework for Advanced Mathematical Reasoning in Large Language Models
  5. 2025Mathematical Computation and Reasoning Errors by Large Language Models
  6. 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
  7. 2024Benchmarking Large Language Models for Math Reasoning Tasks
  8. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges