mlsanityData to 7 Oct 2026

An Empirical Comparison of Pre-Trained Models of Source Code

arXiv 2302.04026 ↗First posted 8 Feb 2023cs.SEcited by 25

Not on the map. Why?

Abstract (arXiv)

While a large number of pre-trained models of source code have been successfully developed and applied to a variety of software engineering (SE) tasks in recent years, our understanding of these pre-trained models is arguably fairly limited. With the goal of advancing our understanding of these models, we perform the first systematic empirical comparison of 19 recently-developed pre-trained models of source code on 13 SE tasks. To gain additional insights into these models, we adopt a recently-developed 4-dimensional categorization of pre-trained models, and subsequently investigate whether there are correlations between different categories of pre-trained models and their performances on different SE tasks.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Feb 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 6; 2025: 7; 2026: 1.

25 papers.

Sort:

By area:

  1. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHubcited by 53
  2. 2023Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generationcited by 49
  3. 2023CodeFusion: A Pre-trained Diffusion Model for Code Generationcited by 31
  4. 2025An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunitiescited by 17
  5. 2023Advising OpenMP Parallelization via a Graph-Based Approach with Transformerscited by 10
  6. 2024RepoMasterEval: Evaluating Code Completion via Real-World Repositoriescited by 10
  7. 2023Pitfalls in Language Models for Code Intelligence: A Taxonomy and Surveycited by 9
  8. 2023Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Codecited by 9
  9. 2023FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categories and Test Code Repaircited by 6
  10. 2023MPI-rical: Data-Driven MPI Distributed Parallelism Assistance with Transformerscited by 5
  11. 2023A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Taskscited by 5
  12. 2023Do Language Models Learn Semantics of Code? A Case Study in Vulnerability Detectioncited by 4
13 more
  1. 2024Do Current Language Models Support Code Intelligence for R Programming Language?cited by 4
  2. 2025MoSE: Hierarchical Self-Distillation Enhances Early Layer Embeddingscited by 2
  3. 2023Redundancy and Concept Analysis for Code-trained Language Modelscited by 1
  4. 2024Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidancecited by 1
  5. 2024Agent-Driven Automatic Software Improvementcited by 1
  6. 2023DocGen: Generating Detailed Parameter Docstrings in Pythoncited by 0
  7. 2024Studying Vulnerable Code Entities in Rcited by 0
  8. 2024Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphscited by 0
  9. 2025Streamlining Admission with LOR Insights: AI-Based Leadership Assessment in Online Master's Programcited by 0
  10. 2025ReDef: Do Code Language Models Truly Understand Code Changes for Just-in-Time Software Defect Prediction?cited by 0
  11. 2025Selecting and Combining Large Language Models for Scalable Code Clone Detectioncited by 0
  12. 2025MANTRA: a Framework for Multi-stage Adaptive Noise TReAtment During Trainingcited by 0
  13. 2026Recall Before Rerank: Benchmarking Deep Learning Models for Large-Scale Code-to-Code Retrievalcited by 0