mlsanityData to 14 Oct 2026

Kotlin ML Pack: Technical Report

arXiv 2405.19250 ↗First posted 29 May 2024cs.SE cs.AI cs.PLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this technical report, we present three novel datasets of Kotlin code: KStack, KStack-clean, and KExercises. We also describe the results of fine-tuning CodeLlama and DeepSeek models on this data. Additionally, we present a version of the HumanEval benchmark rewritten by human experts into Kotlin - both the solutions and the tests. Our results demonstrate that small, high-quality datasets (KStack-clean and KExercises) can significantly improve model performance on code generation tasks, achieving up to a 16-point increase in pass rate on the HumanEval benchmark. Lastly, we discuss potential future work in the field of improving language modeling for Kotlin, including the use of static analysis tools in the learning process and the introduction of more intricate and realistic benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

18 of 25 references.

References · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  7. 2021Measuring Coding Challenge Competence With APPScited by 902
  8. 2023StarCoder: may the source be with you!cited by 799
  9. 2023Textbooks Are All You Needcited by 569
  10. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  11. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  12. 2022The Stack: 3 TB of permissively licensed source codecited by 359
6 more
  1. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  2. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  3. 2023Small-scale proxies for large-scale Transformer training instabilitiescited by 183
  4. 2022Multi-lingual Evaluation of Code Generation Modelscited by 159
  5. 2023Pretraining on the Test Set Is All You Needcited by 33
  6. 2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28

Built on by

Not cited yet.