mlsanityData to 7 Oct 2026

Finding Reusable Machine Learning Components to Build Programming Language Processing Pipelines

arXiv 2208.05596 ↗First posted 11 Aug 2022cs.LG cs.PLcited by 3

Not on the map. Why?

Abstract (arXiv)

Programming Language Processing (PLP) using machine learning has made vast improvements in the past few years. Increasingly more people are interested in exploring this promising field. However, it is challenging for new researchers and developers to find the right components to construct their own machine learning pipelines, given the diverse PLP tasks to be solved, the large number of datasets and models being released, and the set of complex compilers or tools involved. To improve the findability, accessibility, interoperability and reusability (FAIRness) of machine learning components, we collect and analyze a set of representative papers in the domain of machine learning-based PLP. We then identify and characterize key concepts including PLP tasks, model architectures and supportive tools. Finally, we show some example use cases of leveraging the reusable components to construct machine learning pipelines to solve a set of PLP tasks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 11 Aug 2022.

Builds on

34 of 50 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  10. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  11. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  12. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
22 more
  1. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  2. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  3. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
  4. 2020Unsupervised Translation of Programming Languagescited by 262
  5. 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
  6. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
  7. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  8. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  9. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  10. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  11. 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 106
  12. 2018Mapping Language to Code in Programmatic Contextcited by 104
  13. 2021Mixed supervision for surface-defect detection: from weakly to fully supervised learningcited by 101
  14. 2018Tree-to-tree Neural Networks for Program Translationcited by 93
  15. 2020Contrastive Code Representation Learningcited by 76
  16. 2021Scalable and Efficient MoE Training for Multitask Multilingual Modelscited by 76
  17. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 74
  18. 2021SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representationcited by 63
  19. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  20. 2018A Survey on Compiler Autotuning using Machine Learningcited by 40
  21. 2021TreeBERT: A Tree-Based Pre-Trained Model for Programming Languagecited by 40
  22. 2022CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Trainingcited by 15

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.

  1. 2023HPC-GPT: Integrating Large Language Model for High-Performance Computingcited by 22
  2. 2023LM4HPC: Towards Effective Language Model Application in High-Performance Computingcited by 18
  3. 2023Data Race Detection Using Large Language Modelscited by 11