mlsanityData to 14 Oct 2026

A Unified Model for Cross-Domain Clone Detection via Model Merging

arXiv 2608.04215 ↗First posted 4 Aug 2026cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The growing diversity of code clone types, from syntactic copies to cross-language semantic clones to AI-generated duplicates, has created a fragmentation crisis in clone detection. Current deep learning detectors are domain specialists that degrade significantly outside their training distribution, with F1 drops exceeding 70% across domains. Deploying multiple specialized models is impractical, yet training a single cross-domain detector requires simultaneous access to all training data. To address this, we investigate model merging, a family of post-hoc techniques that operate solely on trained checkpoints. We evaluate parameter merging with five task-vector methods, architecture merging via greedy layer stitching, and cross-tokenizer alignment across four code models, three benchmarks, and twelve configurations. Same-base TIES merging creates effective cross-domain detectors, validated across two model families and three random seeds, reaching 0.865 combined F1 on UniXcoder, 93% of multi-task performance without any training data at the merging step. WUDI achieves the highest in-distribution combined F1 at 0.899, but TIES generalizes better to unseen AI-generated clones, making it our recommended method. Cross-base merging yields only marginal and high-variance gains across all five methods, indicating that task vector compatibility through a shared pre-trained base is the binding factor for effective merging. Merged detectors also outperform zero-shot code LLMs on GPTCloneBench at lower inference cost and generalize up to 4x better than multi-task training to unseen AI-generated clones, suggesting a trade-off between in-domain performance and OOD robustness. This work provides one of the first systematic empirical studies of model merging for software engineering and a practical recipe for building cross-domain clone detectors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 67 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

28 of 67 references.

References · 28

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  4. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  5. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  6. 2022Editing Models with Task Arithmeticcited by 1,120
  7. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  8. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  9. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  10. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  11. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
16 more
  1. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 375
  2. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  3. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  4. 2024Parameter Competition Balancing for Model Mergingcited by 72
  5. 2020Detecting Code Clones with Graph Neural Networkand Flow-Augmented Abstract Syntax Treecited by 69
  6. 2024Learning to Route Among Specialized Experts for Zero-Shot Generalizationcited by 65
  7. 2025Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectorscited by 62
  8. 2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
  9. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  10. 2023Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Surveycited by 15
  11. 2023GPTCloneBench: A comprehensive benchmark of semantic clones and cross-language clones using GPT-3 model and SemanticCloneBenchcited by 11
  12. 2024HM3: Hierarchical Multi-Objective Model Merging for Pretrained Modelscited by 8
  13. 2021Clones in Deep Learning Code: What, Where, and Why?cited by 5
  14. 2023Unveiling the potential of large language models in generating semantic and cross-language clonescited by 3
  15. 2024Deep Learning-Based Out-of-distribution Source Code Data Identification: How Far Have We Gone?cited by 2
  16. 2025Are Classical Clone Detectors Good Enough For the AI Era?cited by 2

Built on by

Not cited yet.