mlsanityData to 14 Oct 2026

A Comprehensive Review of State-of-The-Art Methods for Java Code Generation from Natural Language Text

arXiv 2306.06371 ↗First posted 10 Jun 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Java Code Generation consists in generating automatically Java code from a Natural Language Text. This NLP task helps in increasing programmers' productivity by providing them with immediate solutions to the simplest and most repetitive tasks. Code generation is a challenging task because of the hard syntactic rules and the necessity of a deep understanding of the semantic aspect of the programming language. Many works tried to tackle this task using either RNN-based, or Transformer-based models. The latter achieved remarkable advancement in the domain and they can be divided into three groups: (1) encoder-only models, (2) decoder-only models, and (3) encoder-decoder models. In this paper, we provide a comprehensive review of the evolution and progress of deep learning models in Java code generation task. We focus on the most important methods and present their merits and limitations, as well as the objective functions used by the community. In addition, we provide a detailed description of datasets and evaluation metrics used in the literature. Finally, we discuss results of different models on CONCODE dataset, then propose some future directions.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 53 of its 86 references that are in the map. First posted on arXiv 10 Jun 2023.

Builds on

53 of 86 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  11. 2016Layer Normalizationcited by 5,669
  12. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
38 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  3. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  4. 2019Root Mean Square Layer Normalizationcited by 1,480
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  8. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  10. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  11. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  12. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  13. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  14. 2014Neural Turing Machinescited by 644
  15. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  16. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  17. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  18. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  19. 2021Unified Pre-training for Program Understanding and Generationcited by 366
  20. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  21. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  22. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  23. 2021Scaling Laws for Transfercited by 272
  24. 2020Unsupervised Translation of Programming Languagescited by 263
  25. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 218
  26. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  27. 2021Retrieval Augmented Code Generation and Summarizationcited by 149
  28. 2015Ask Me Anything: Dynamic Memory Networks for Natural Language Processingcited by 117
  29. 2018Mapping Language to Code in Programmatic Contextcited by 104
  30. 2015Neural Programmer-Interpreterscited by 101
  31. 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
  32. 2015Inferring Algorithmic Patterns with Stack-Augmented Recurrent Netscited by 76
  33. 2016Latent Predictor Networks for Code Generationcited by 73
  34. 2021DOBF: A Deobfuscation Pre-Training Objective for Programming Languagescited by 68
  35. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  36. 2016Deep API Learningcited by 41
  37. 2015Neural Programmer: Inducing Latent Programs with Gradient Descentcited by 34
  38. 2022StructCoder: Structure-Aware Transformer for Code Generationcited by 34

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.