# Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization

arXiv: https://arxiv.org/abs/2207.05579 · HTML: https://mlsanity.com/paper/2207.05579/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2022.

## Built on by (most cited first, 22 of 22)

- [Distilled GPT for Source Code Summarization](https://mlsanity.com/paper/2308.14731/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation](https://mlsanity.com/paper/2305.06156/) (2023)
- [Data Cleaning and Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2310.01765/) (2023)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [The Fault in our Stars: Quality Assessment of Code Generation Benchmarks](https://mlsanity.com/paper/2404.10155/) (2024)
- [ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization](https://mlsanity.com/paper/2407.01646/) (2024)
- [XMainframe: A Large Language Model for Mainframe Modernization](https://mlsanity.com/paper/2408.04660/) (2024)
- [Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks](https://mlsanity.com/paper/2504.19444/) (2025)
- [Semantic Similarity Loss for Neural Source Code Summarization](https://mlsanity.com/paper/2308.07429/) (2023)
- [DocChecker: Bootstrapping Code Large Language Model for Detecting and Resolving Code-Comment Inconsistencies](https://mlsanity.com/paper/2306.06347/) (2023)
- [Revisiting File Context for Source Code Summarization](https://mlsanity.com/paper/2309.02326/) (2023)
- [Commenting Higher-level Code Unit: Full Code, Reduced Code, or Hierarchical Code Summarization](https://mlsanity.com/paper/2503.10737/) (2025)
- [MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment Generation](https://mlsanity.com/paper/2403.17357/) (2024)
- [CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement](https://mlsanity.com/paper/2505.19757/) (2025)
- [TREAT: A Code LLMs Trustworthiness / Reliability Evaluation and Testing Framework](https://mlsanity.com/paper/2510.17163/) (2025)
- [Automated Summarization of Software Documents: An LLM-based Multi-Agent Approach](https://mlsanity.com/paper/2606.24689/) (2026)
- [AI-Mediated Code Comment Improvement](https://mlsanity.com/paper/2505.09021/) (2025)
- [Enhancing Neural Code Representation with Additional Context](https://mlsanity.com/paper/2510.12082/) (2025)
- [Toward Explaining Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2512.20328/) (2025)
- [On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Study](https://mlsanity.com/paper/2601.23059/) (2026)
- [On the Lexical Superstition of Large Language Models for Code Comprehension: Re-evaluation on Code of Low Lexical Quality](https://mlsanity.com/paper/2609.26388/) (2026)

## Abstract (arXiv)

Code summarization, the task of generating useful comments given the code, has long been of interest. Most of the existing code summarization models are trained and validated on widely-used code comment benchmark datasets. However, little is known about the quality of the benchmark datasets built from real-world projects. Are the benchmark datasets as good as expected? To bridge the gap, we conduct a systematic research to assess and improve the quality of four benchmark datasets widely used for code summarization tasks. First, we propose an automated code-comment cleaning tool that can accurately detect noisy data caused by inappropriate data preprocessing operations from existing benchmark datasets. Then, we apply the tool to further assess the data quality of the four benchmark datasets, based on the detected noises. Finally, we conduct comparative experiments to investigate the impact of noisy data on the performance of code summarization models. The results show that these data preprocessing noises widely exist in all four benchmark datasets, and removing these noisy data leads to a significant improvement on the performance of code summarization. We believe that the findings and insights will enable a better understanding of data quality in code summarization tasks, and pave the way for relevant research and practice.