JEMMA: An Extensible Java Dataset for ML4Code Applications
Not on the map. Why?
Abstract (arXiv)
Machine Learning for Source Code (ML4Code) is an active research field in which extensive experimentation is needed to discover how to best use source code's richly structured information. With this in mind, we introduce JEMMA, an Extensible Java Dataset for ML4Code Applications, which is a large-scale, diverse, and high-quality dataset targeted at ML4Code. Our goal with JEMMA is to lower the barrier to entry in ML4Code by providing the building blocks to experiment with source code models and tasks. JEMMA comes with a considerable amount of pre-processed information such as metadata, representations (e.g., code tokens, ASTs, graphs), and several properties (e.g., metrics, static analysis results) for 50,000 Java projects from the 50KC dataset, with over 1.2 million classes and over 8 million methods. JEMMA is also extensible allowing users to add new properties and representations to the dataset, and evaluate tasks on them. Thus, JEMMA becomes a workbench that researchers can use to experiment with novel representations and tasks operating on source code. To demonstrate the utility of the dataset, we also report results from two empirical studies on our data, ultimately showing that significant work lies ahead in the design of context-aware source code models that can reason over a broader network of source code entities in a software project, the very task that JEMMA is designed to help with.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 52 of its 109 references that are in the map. First posted on arXiv 18 Dec 2022.
Builds on
52 of 109 references.
References · 52
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2020Reformer: The Efficient Transformercited by 1,658
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
- 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
- 2020Efficient Transformers: A Surveycited by 784
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
- 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
38 more
- 2018Annotation Artifacts in Natural Language Inference Datacited by 656
- 2018Recognition in Terra Incognitacited by 655
- 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
- 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
- 2021Unified Pre-training for Program Understanding and Generationcited by 361
- 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
- 2018code2vec: Learning Distributed Representations of Codecited by 249
- 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
- 2017Learning to Represent Programs with Graphscited by 219
- 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
- 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
- 2018The Adverse Effects of Code Duplication in Machine Learning Models of Codecited by 123
- 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 117
- 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 106
- 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 97
- 2019A Neural Model for Generating Natural Language Summaries of Program Subroutinescited by 76
- 2018DeepBugs: A Learning Approach to Name-based Bug Detectioncited by 65
- 2017A parallel corpus of Python functions and documentation strings for automated code documentation and code generationcited by 56
- 2020Big Code != Big Vocabulary: Open-Vocabulary Models for Source Codecited by 53
- 2019Recommendations for Datasets for Source Code Summarizationcited by 46
- 2020LambdaNet: Probabilistic Type Inference using Graph Neural Networkscited by 44
- 2015SourcererCC: Scaling Code Clone Detection to Big Codecited by 40
- 2019Neural Program Repair by Jointly Learning to Localize and Repaircited by 39
- 2019How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Datasetcited by 35
- 2018StaQC: A Systematically Mined Question-Code Dataset from Stack Overflowcited by 30
- 2020Typilus: Neural Type Hintscited by 29
- 2021Project-Level Encoding for Neural Source Code Summarization of Subroutinescited by 24
- 2021Automatic Program Repair with OpenAI's Codex: Evaluating QuixBugscited by 24
- 2021An Empirical Study on the Usage of BERT Models for Code Completioncited by 22
- 2018NAPS: Natural Program Synthesis Datasetcited by 14
- 2021Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchycited by 14
- 2019COSET: A Benchmark for Evaluating Neural Program Embeddingscited by 10
- 2021ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inferencecited by 9
- 2018Path-Based Function Embedding and its Application to Specification Miningcited by 8
- 2020World of Code: Enabling a Research Workflow for Mining and Analyzing the Universe of Open Source VCS datacited by 7
- 2022Learning to Recommend Method Names with Global Contextcited by 6
- 2021A Context-based Automated Approach for Method Name Consistency Checking and Suggestioncited by 5
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2023: 2.
2 papers.