mlsanityData to 7 Oct 2026

Automating Code Review Activities by Large-Scale Pre-training

arXiv 2203.09095 ↗First posted 17 Mar 2022cs.SE cs.AIcited by 98

Not on the map. Why?

Abstract (arXiv)

Code review is an essential part to software development lifecycle since it aims at guaranteeing the quality of codes. Modern code review activities necessitate developers viewing, understanding and even running the programs to assess logic, functionality, latency, style and other factors. It turns out that developers have to spend far too much time reviewing the code of their peers. Accordingly, it is in significant demand to automate the code review process. In this research, we focus on utilizing pre-training techniques for the tasks in the code review scenario. We collect a large-scale dataset of real-world code changes and code reviews from open-source projects in nine of the most popular programming languages. To better understand code diffs and reviews, we propose CodeReviewer, a pre-trained model that utilizes four pre-training tasks tailored specifically for the code review scenario. To evaluate our model, we focus on three key tasks related to code review activities, including code change quality estimation, review comment generation and code refinement. Furthermore, we establish a high-quality benchmark dataset based on our collected data for these three tasks and conduct comprehensive experiments on it. The experimental results demonstrate that our model outperforms the previous state-of-the-art pre-training approaches in all tasks. Further analysis show that our proposed pre-training tasks and the multilingual pre-training dataset benefit the model on the understanding of code changes and reviews.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Builds on 22 of its 50 references that are in the map. First posted on arXiv 17 Mar 2022.

Builds on

22 of 50 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  11. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
10 more
  1. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  2. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
  3. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  4. 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 106
  5. 2022Using Pre-Trained Models to Boost Code Review Automationcited by 63
  6. 2021Towards Automating Code Review Activitiescited by 54
  7. 2020A Systematic Literature Review on the Use of Deep Learning in Software Engineering Researchcited by 35
  8. 2019CORE: Automating Review Recommendation for Code Changescited by 14
  9. 2020Deep Learning & Software Engineering: State of Research and Future Directionscited by 9
  10. 2021Breaking Down Multilingual Machine Translationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 15; 2024: 19; 2025: 30; 2026: 33.

98 papers.

Sort:

By area:

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  3. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  4. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  5. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  6. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  7. 2024NExT: Teaching Large Language Models to Reason about Code Executioncited by 60
  8. 2023LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuningcited by 47
  9. 2023Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructionscited by 46
  10. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
  11. 2024Automatic Programming: Large Language Models and Beyondcited by 35
  12. 2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
38 more
  1. 2024History, Development, and Principles of Large Language Models-An Introductory Surveycited by 30
  2. 2024NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctnesscited by 23
  3. 2024Unsupervised Evaluation of Code LLMs with Round-Trip Correctnesscited by 23
  4. 2023From Commit Message Generation to History-Aware Commit Message Completioncited by 16
  5. 2024AI-Assisted Assessment of Coding Practices in Modern Code Reviewcited by 16
  6. 2023FLAME: A small language model for spreadsheet formulascited by 15
  7. 2023Towards Generating Functionally Correct Code Edits from Natural Language Issue Descriptionscited by 14
  8. 2025CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generationcited by 14
  9. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  10. 2023Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language Modelscited by 11
  11. 2023Multilingual Code Co-Evolution Using Large Language Modelscited by 9
  12. 2023Automating Code-Related Tasks Through Transformers: The Impact of Pre-trainingcited by 8
  13. 2025LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLMcited by 8
  14. 2024Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generationcited by 7
  15. 2024CRQBench: A Benchmark of Code Reasoning Questionscited by 6
  16. 2025Combining Large Language Models with Static Analyzers for Code Review Generationcited by 6
  17. 2025REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinementcited by 6
  18. 2025E2Edev: Benchmarking Large Language Models in End-to-End Software Development Taskcited by 6
  19. 2026RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassiancited by 6
  20. 2025SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuningcited by 5
  21. 2026AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Contextcited by 5
  22. 2023Configuration Validation with Large Language Modelscited by 4
  23. 2024Leveraging Reviewer Experience in Code Review Comment Generationcited by 4
  24. 2025Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Modelscited by 4
  25. 2025Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metricscited by 4
  26. 2025Issue-Oriented Agent-Based Framework for Automated Review Comment Generationcited by 4
  27. 2026Understanding Dominant Themes in Reviewing Agentic AI-authored Codecited by 4
  28. 2026Code Review Agent Benchmarkcited by 4
  29. 2024Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trendscited by 3
  30. 2024Beyond the Sum: Unlocking AI Agents Potential Through Market Forcescited by 3
  31. 2025Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Qualitycited by 3
  32. 2025CodeReviewQA: The Code Review Comprehension Assessment for Large Language Modelscited by 3
  33. 2025Towards Practical Defect-Focused Automated Code Reviewcited by 3
  34. 2025Towards Transparent AI: A Survey on Explainable Language Modelscited by 3
  35. 2025EduAdapt: A Question Answer Benchmark Dataset for Evaluating Grade-Level Adaptability in LLMscited by 3
  36. 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Reviewcited by 3
  37. 2024PatchFinder: A Two-Phase Approach to Security Patch Tracing for Disclosed Vulnerabilities in Open-Source Softwarecited by 2
  38. 2025LAMS: LLM-Driven Automatic Mode Switching for Assistive Teleoperationcited by 2

50 most cited of 98 All 98 (JSON)