mlsanityData to 14 Oct 2026

Can Programming Languages Boost Each Other via Instruction Tuning?

arXiv 2308.16824 ↗First posted 31 Aug 2023cs.CL cs.AI cs.PLcited by 7

Not on the map. Why?

Abstract (arXiv)

When human programmers have mastered a programming language, it would be easier when they learn a new programming language. In this report, we focus on exploring whether programming languages can boost each other during the instruction fine-tuning phase of code large language models. We conduct extensive experiments of 8 popular programming languages (Python, JavaScript, TypeScript, C, C++, Java, Go, HTML) on StarCoder. Results demonstrate that programming languages can significantly improve each other. For example, CodeM-Python 15B trained on Python is able to increase Java by an absolute 17.95% pass@1 on HumanEval-X. More surprisingly, we found that CodeM-HTML 7B trained on the HTML corpus can improve Java by an absolute 15.24% pass@1. Our training data is released at https://github.com/NL2Code/CodeM.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 16 of its 17 references that are in the map. First posted on arXiv 31 Aug 2023.

Builds on

16 of 17 references.

References · 16

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  8. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  9. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  10. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  11. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  12. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
4 more
  1. 2023SantaCoder: don't reach for the stars!cited by 170
  2. 2022Large Language Models Meet NL2Code: A Surveycited by 138
  3. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  4. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69

Built on by

Cited by year

Citing papers per year: 2024: 7.

7 papers.

  1. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  2. 2024UniCoder: Scaling Code Large Language Model via Universal Codecited by 18
  3. 2024The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Modelscited by 6
  4. 2024SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Modelscited by 6
  5. 2024Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspectivecited by 3
  6. 2024Improving Long Text Understanding with Knowledge Distilled from Summarization Modelcited by 3
  7. 2024Raw Text is All you Need: Knowledge-intensive Multi-turn Instruction Tuning for Large Language Modelcited by 3