A Comprehensive Review of State-of-The-Art Methods for Java Code Generation from Natural Language Text
Not on the map. Why?
Abstract (arXiv)
Java Code Generation consists in generating automatically Java code from a Natural Language Text. This NLP task helps in increasing programmers' productivity by providing them with immediate solutions to the simplest and most repetitive tasks. Code generation is a challenging task because of the hard syntactic rules and the necessity of a deep understanding of the semantic aspect of the programming language. Many works tried to tackle this task using either RNN-based, or Transformer-based models. The latter achieved remarkable advancement in the domain and they can be divided into three groups: (1) encoder-only models, (2) decoder-only models, and (3) encoder-decoder models. In this paper, we provide a comprehensive review of the evolution and progress of deep learning models in Java code generation task. We focus on the most important methods and present their merits and limitations, as well as the objective functions used by the community. In addition, we provide a detailed description of datasets and evaluation metrics used in the literature. Finally, we discuss results of different models on CONCODE dataset, then propose some future directions.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 53 of its 86 references that are in the map. First posted on arXiv 10 Jun 2023.
Builds on
53 of 86 references.
References · 53
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2016Layer Normalizationcited by 5,669
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
38 more
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
- 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
- 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
- 2014Neural Turing Machinescited by 644
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
- 2022Towards Reasoning in Large Language Models: A Surveycited by 596
- 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
- 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
- 2021Unified Pre-training for Program Understanding and Generationcited by 366
- 2022A Systematic Evaluation of Large Language Models of Codecited by 363
- 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
- 2021Scaling Laws for Transfercited by 272
- 2020Unsupervised Translation of Programming Languagescited by 263
- 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 218
- 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
- 2021Retrieval Augmented Code Generation and Summarizationcited by 149
- 2015Ask Me Anything: Dynamic Memory Networks for Natural Language Processingcited by 117
- 2018Mapping Language to Code in Programmatic Contextcited by 104
- 2015Neural Programmer-Interpreterscited by 101
- 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
- 2015Inferring Algorithmic Patterns with Stack-Augmented Recurrent Netscited by 76
- 2016Latent Predictor Networks for Code Generationcited by 73
- 2021DOBF: A Deobfuscation Pre-Training Objective for Programming Languagescited by 68
- 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
- 2016Deep API Learningcited by 41
- 2015Neural Programmer: Inducing Latent Programs with Gradient Descentcited by 34
- 2022StructCoder: Structure-Aware Transformer for Code Generationcited by 34
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.