CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
Not on the map. Why?
Abstract (arXiv)
Effective generation of structured code comments requires robust quality metrics for dataset curation, yet existing approaches (SIDE, MIDQ, STASIS) suffer from limited code-comment analysis. We propose CIDRe, a language-agnostic reference-free quality criterion combining four synergistic aspects: (1) relevance (code-comment semantic alignment), (2) informativeness (functional coverage), (3) completeness (presence of all structure sections), and (4) description length (detail sufficiency). We validate our criterion on a manually annotated dataset. Experiments demonstrate CIDRe's superiority over existing metrics, achieving improvement in cross-entropy evaluation. When applied to filter comments, the models finetuned on CIDRe-filtered data show statistically significant quality gains in GPT-4o-mini assessments.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 26 references that are in the map. First posted on arXiv 26 May 2025.
Builds on
15 of 26 references.
References · 15
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
- 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
- 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
- 2022The Stack: 3 TB of permissively licensed source codecited by 359
- 2019Recommendations for Datasets for Source Code Summarizationcited by 46
- 2022MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languagescited by 41
- 2024CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrievalcited by 34
- 2022Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarizationcited by 22
3 more
- 2023Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterizationcited by 21
- 2023The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generationcited by 20
- 2025Optimizing Datasets for Code Summarization: Is Code-Comment Coherence Enough?cited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.