# CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement

arXiv: https://arxiv.org/abs/2505.19757 · HTML: https://mlsanity.com/paper/2505.19757/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Recommendations for Datasets for Source Code Summarization](https://mlsanity.com/paper/1904.02660/) (2019)
- [MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages](https://mlsanity.com/paper/2203.08388/) (2022)
- [CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval](https://mlsanity.com/paper/2411.12644/) (2024)
- [Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization](https://mlsanity.com/paper/2207.05579/) (2022)
- [Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterization](https://mlsanity.com/paper/2312.15475/) (2023)
- [The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation](https://mlsanity.com/paper/2305.06156/) (2023)
- [Optimizing Datasets for Code Summarization: Is Code-Comment Coherence Enough?](https://mlsanity.com/paper/2502.07611/) (2025)

## Built on by (most cited first, 1 of 1)

- [Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code](https://mlsanity.com/paper/2605.05267/) (2026)

## Abstract (arXiv)

Effective generation of structured code comments requires robust quality metrics for dataset curation, yet existing approaches (SIDE, MIDQ, STASIS) suffer from limited code-comment analysis. We propose CIDRe, a language-agnostic reference-free quality criterion combining four synergistic aspects: (1) relevance (code-comment semantic alignment), (2) informativeness (functional coverage), (3) completeness (presence of all structure sections), and (4) description length (detail sufficiency). We validate our criterion on a manually annotated dataset. Experiments demonstrate CIDRe's superiority over existing metrics, achieving improvement in cross-entropy evaluation. When applied to filter comments, the models finetuned on CIDRe-filtered data show statistically significant quality gains in GPT-4o-mini assessments.