# Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance

arXiv: https://arxiv.org/abs/2404.08817 · HTML: https://mlsanity.com/paper/2404.08817/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Builds on (most cited first, 9 of 9)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow](https://mlsanity.com/paper/1805.08949/) (2018)
- [Multi-lingual Evaluation of Code Generation Models](https://mlsanity.com/paper/2210.14868/) (2022)
- [Is ChatGPT the Ultimate Programming Assistant -- How far is it?](https://mlsanity.com/paper/2304.11938/) (2023)
- [Enhancing Text-to-SQL Translation for Financial System Design](https://mlsanity.com/paper/2312.14725/) (2023)

## Built on by (most cited first, 26 of 26)

- [CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair](https://mlsanity.com/paper/2409.12993/) (2024)
- [cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree](https://mlsanity.com/paper/2506.15655/) (2025)
- [Sherlock: Reliable and Efficient Agentic Workflow Execution](https://mlsanity.com/paper/2511.00330/) (2025)
- [Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization](https://mlsanity.com/paper/2412.04619/) (2024)
- [Detection of LLM-Paraphrased Code and Identification of the Responsible LLM Using Coding Style Features](https://mlsanity.com/paper/2502.17749/) (2025)
- [Code Roulette: How Prompt Variability Affects LLM Code Generation](https://mlsanity.com/paper/2506.10204/) (2025)
- [Dynamic Stability of LLM-Generated Code](https://mlsanity.com/paper/2511.07463/) (2025)
- [Correctness isnt Efficiency: Runtime Memory Divergence in LLM-Generated Code](https://mlsanity.com/paper/2601.01215/) (2026)
- [CodeRefine: A Pipeline for Enhancing LLM-Generated Code Implementations of Research Papers](https://mlsanity.com/paper/2408.13366/) (2024)
- [LLM-Supported Natural Language to Bash Translation](https://mlsanity.com/paper/2502.06858/) (2025)
- [OpenClassGen: A Large-Scale Corpus of Real-World Python Classes for LLM Research](https://mlsanity.com/paper/2504.15564/) (2025)
- [TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Design](https://mlsanity.com/paper/2601.21239/) (2026)
- [NES: An Instruction-Free, Low-Latency Next Edit Suggestion Framework Powered by Learned Historical Editing Trajectories](https://mlsanity.com/paper/2508.02473/) (2025)
- [Measuring LLM Code Generation Stability via Structural Entropy](https://mlsanity.com/paper/2508.14288/) (2025)
- [SLMFix: Leveraging Small Language Models for Domain Specific Language Error Fixing with Reinforcement Learning](https://mlsanity.com/paper/2511.19422/) (2025)
- [Beyond Query-Level Comparison: Fine-Grained Reinforcement Learning for Text-to-SQL with Automated Interpretable Critiques](https://mlsanity.com/paper/2511.22258/) (2025)
- [CAP: Data Contamination Detection via Consistency Amplification](https://mlsanity.com/paper/2410.15005/) (2024)
- [How Small Transformation Expose the Weakness of Semantic Similarity Measures](https://mlsanity.com/paper/2509.09714/) (2025)
- [CSSG: Measuring Code Similarity with Semantic Graphs](https://mlsanity.com/paper/2601.04085/) (2026)
- [MATA: Multi-Agent Framework for Reliable and Flexible Table Question Answering](https://mlsanity.com/paper/2602.09642/) (2026)
- [Breaking Validity-Induced Boundaries to Expand Algorithm Search Space: A Two-Stage AST-Based Operator for LLM-Driven Automated Heuristic Evolution](https://mlsanity.com/paper/2604.16420/) (2026)
- [Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiency](https://mlsanity.com/paper/2608.14065/) (2026)
- [Skeletons Matter: Dynamic Data Augmentation for Text-to-Query](https://mlsanity.com/paper/2511.18934/) (2025)
- [Structure-BiEval: A Self-Supervised, Dual-Track Framework for Decoupling Structure and Content in LLM Evaluation for Web Information Systems](https://mlsanity.com/paper/2601.19923/) (2026)
- [Can Code Evaluation Metrics Detect Code Plagiarism?](https://mlsanity.com/paper/2604.25778/) (2026)
- [Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation](https://mlsanity.com/paper/2606.28593/) (2026)

## Abstract (arXiv)

This paper revisits recent code similarity evaluation metrics, particularly focusing on the application of Abstract Syntax Tree (AST) editing distance in diverse programming languages. In particular, we explore the usefulness of these metrics and compare them to traditional sequence similarity metrics. Our experiments showcase the effectiveness of AST editing distance in capturing intricate code structures, revealing a high correlation with established metrics. Furthermore, we explore the strengths and weaknesses of AST editing distance and prompt-based GPT similarity scores in comparison to BLEU score, execution match, and Jaccard Similarity. We propose, optimize, and publish an adaptable metric that demonstrates effectiveness across all tested languages, representing an enhanced version of Tree Similarity of Edit Distance (TSED).