mlsanityData to 14 Oct 2026

Narrow Transformer: StarCoder-Based Java-LM For Desktop

arXiv 2407.03941 ↗First posted 4 Jul 2024cs.SE cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents NT-Java-1.1B, an open-source specialized code language model built on StarCoderBase-1.1B, designed for coding tasks in Java programming. NT-Java-1.1B achieves state-of-the-art performance, surpassing its base model and majority of other models of similar size on MultiPL-E Java code benchmark. While there have been studies on extending large, generic pre-trained models to improve proficiency in specific programming languages like Python, similar investigations on small code models for other programming languages are lacking. Large code models require specialized hardware like GPUs for inference, highlighting the need for research into building small code models that can be deployed on developer desktops. This paper addresses this research gap by focusing on the development of a small Java code model, NT-Java-1.1B, and its quantized versions, which performs comparably to open models around 1.1B on MultiPL-E Java code benchmarks, making them ideal for desktop deployment. This paper establishes the foundation for specialized models across languages and sizes for a family of NT Models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 13 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

13 of 13 references.

References · 13

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  5. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  6. 2023StarCoder: may the source be with you!cited by 799
  7. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  8. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  9. 2023Textbooks Are All You Needcited by 569
  10. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  11. 2023SantaCoder: don't reach for the stars!cited by 170
  12. 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 136
1 more
  1. 2023CodeGen2: Lessons for Training LLMs on Programming and Natural Languagescited by 125

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024MojoBench: Language Modeling and Benchmarks for Mojocited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025REMODEL-LLM: Transforming C code to Java using LLMs
  2. 2024An Empirical Study on the Code Refactoring Capability of Large Language Models
  3. 2026Quantize with Confidence? An Empirical Study of Quantization for Code Generation
  4. 2025Holistic Evaluation of State-of-the-Art LLMs for Code Generation
  5. 2026Compressed code: the hidden effects of quantization and distillation on programming tokens
  6. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
  8. 2025CWM: An Open-Weights LLM for Research on Code Generation with World Models