mlsanityData to 13 Oct 2026

C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Pooling

arXiv 2512.21332 ↗First posted 24 Dec 2025cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

We present C2LLM - Contrastive Code Large Language Models, a family of code embedding models in both 0.5B and 7B sizes. Building upon Qwen-2.5-Coder backbones, C2LLM adopts a Pooling by Multihead Attention (PMA) module for generating sequence embedding from token embeddings, effectively 1) utilizing the LLM's causal representations acquired during pretraining, while also 2) being able to aggregate information from all tokens in the sequence, breaking the information bottleneck in EOS-based sequence embeddings, and 3) supporting flexible adaptation of embedding dimension, serving as an alternative to MRL. Trained on three million publicly available data, C2LLM models set new records on MTEB-Code among models of similar sizes, with C2LLM-7B ranking 1st on the overall leaderboard.

Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 35 of its 38 references that are in the map. First posted on arXiv 24 Dec 2025.

Builds on

35 of 38 references.

References · 35

  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,570
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,676
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,304
  4. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,222
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,179
  6. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,058
  7. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 977
  8. 2021Measuring Coding Challenge Competence With APPScited by 891
  9. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 743
  10. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 651
  11. 2022MTEB: Massive Text Embedding Benchmarkcited by 630
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
23 more
  1. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 467
  2. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  3. 2022Matryoshka Representation Learningcited by 274
  4. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 243
  5. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 184
  6. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 132
  7. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 121
  8. 2025Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scalingcited by 93
  9. 2024Making Text Embedders Few-Shot Learnerscited by 79
  10. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 75
  11. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  12. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  13. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 56
  14. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  15. 2024CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrievalcited by 33
  16. 2025Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Taskscited by 27
  17. 2024Code Representation Learning At Scalecited by 26
  18. 2024E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoningcited by 13
  19. 2024D2LLM: Decomposed and Distilled Large Language Models for Semantic Searchcited by 9
  20. 2025Towards A Generalist Code Embedding Model Based On Massive Data Synthesiscited by 8
  21. 2025F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Datacited by 8
  22. 2025LGAI-EMBEDDING-Preview Technical Reportcited by 3
  23. 2025QZhou-Embedding Technical Reportcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual Worldcited by 12
  2. 2026CORE-Bench: A Comprehensive Benchmark for Code Retrieval in the Era of Agentic Codingcited by 7
  3. 2026Beyond Retrieval: A Multitask Benchmark and Model for Code Searchcited by 1