# Deep Just-In-Time Inconsistency Detection Between Comments and Source Code

arXiv: https://arxiv.org/abs/2010.01625 · HTML: https://mlsanity.com/paper/2010.01625/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2020.

## Built on by (most cited first, 20 of 20)

- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [On the Evaluation of Neural Code Summarization](https://mlsanity.com/paper/2107.07112/) (2021)
- [CoditT5: Pretraining for Source Code and Natural Language Editing](https://mlsanity.com/paper/2208.05446/) (2022)
- [CodeAgent: Autonomous Communicative Agents for Code Review](https://mlsanity.com/paper/2402.02172/) (2024)
- [Learning Deep Semantics for Test Completion](https://mlsanity.com/paper/2302.10166/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration](https://mlsanity.com/paper/2603.21019/) (2026)
- [Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM](https://mlsanity.com/paper/2503.17793/) (2025)
- [Impact of Evaluation Methodologies on Code Summarization](https://mlsanity.com/paper/2108.09619/) (2021)
- [SciCoQA: Quality Assurance for Scientific Paper--Code Alignment](https://mlsanity.com/paper/2601.12910/) (2026)
- [Code Comment Inconsistency Detection with BERT and Longformer](https://mlsanity.com/paper/2207.14444/) (2022)
- [Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks](https://mlsanity.com/paper/2504.19444/) (2025)
- [CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back](https://mlsanity.com/paper/2302.03924/) (2023)
- [DocChecker: Bootstrapping Code Large Language Model for Detecting and Resolving Code-Comment Inconsistencies](https://mlsanity.com/paper/2306.06347/) (2023)
- [Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding](https://mlsanity.com/paper/2608.11095/) (2026)
- [An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code](https://mlsanity.com/paper/2307.02443/) (2023)
- [CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection](https://mlsanity.com/paper/2511.19875/) (2025)
- [Do Papers Tell the Whole Story? A Benchmark and Framework for Uncovering Hidden Implementation Gaps in Bioinformatics](https://mlsanity.com/paper/2603.22018/) (2026)
- [SkillConsist: Detecting Inconsistencies in Agent Skills via Bidirectional Graph Alignment](https://mlsanity.com/paper/2608.07639/) (2026)
- [Dude: A Dual-Detection Multi-Agent System for Paper-Code Discrepancy Detection](https://mlsanity.com/paper/2609.03416/) (2026)

## Abstract (arXiv)

Natural language comments convey key aspects of source code such as implementation, usage, and pre- and post-conditions. Failure to update comments accordingly when the corresponding code is modified introduces inconsistencies, which is known to lead to confusion and software bugs. In this paper, we aim to detect whether a comment becomes inconsistent as a result of changes to the corresponding body of code, in order to catch potential inconsistencies just-in-time, i.e., before they are committed to a code base. To achieve this, we develop a deep-learning approach that learns to correlate a comment with code changes. By evaluating on a large corpus of comment/code pairs spanning various comment types, we show that our model outperforms multiple baselines by significant margins. For extrinsic evaluation, we show the usefulness of our approach by combining it with a comment update model to build a more comprehensive automatic comment maintenance system which can both detect and resolve inconsistent comments based on code changes.