mlsanityData to 7 Oct 2026

XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence

arXiv 2206.08474 ↗First posted 16 Jun 2022cs.SE cs.AI cs.LGcited by 54

Not on the map. Why?

Abstract (arXiv)

Recent advances in machine learning have significantly improved the understanding of source code data and achieved good performance on a number of downstream tasks. Open source repositories like GitHub enable this process with rich unlabeled code data. However, the lack of high quality labeled data has largely hindered the progress of several code related tasks, such as program translation, summarization, synthesis, and code search. This paper introduces XLCoST, Cross-Lingual Code SnippeT dataset, a new benchmark dataset for cross-lingual code intelligence. Our dataset contains fine-grained parallel data from 8 languages (7 commonly used programming languages and English), and supports 10 cross-lingual code tasks. To the best of our knowledge, it is the largest parallel dataset for source code both in terms of size and the number of languages. We also provide the performance of several state-of-the-art baseline models for each task. We believe this new dataset can be a valuable asset for the research community and facilitate the development and validation of new methods for cross-lingual code intelligence.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 15 of its 20 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

15 of 20 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  8. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  9. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  10. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  11. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  12. 2021Unified Pre-training for Program Understanding and Generationcited by 361
3 more
  1. 2020Unsupervised Translation of Programming Languagescited by 262
  2. 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
  3. 2021AVATAR: A Parallel Corpus for Java-Python Program Translationcited by 59

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 15; 2024: 19; 2025: 16; 2026: 2.

54 papers.

Sort:

By area:

  1. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  2. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  3. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  4. 2022Multi-lingual Evaluation of Code Generation Modelscited by 158
  5. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  6. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  7. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  8. 2024CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferencescited by 56
  9. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 55
  10. 2023xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrievalcited by 46
  11. 2024AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodologycited by 45
  12. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
38 more
  1. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39
  2. 2024CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Executioncited by 31
  3. 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 28
  4. 2024RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translationcited by 21
  5. 2024CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding & Reasoning Capabilities of CodeLLMscited by 18
  6. 2025An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunitiescited by 17
  7. 2023CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Modelscited by 16
  8. 2025Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizationscited by 16
  9. 2024VersiCode: Towards Version-controllable Code Generationcited by 14
  10. 2024CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agentscited by 11
  11. 2023Program Translation via Code Distillationcited by 10
  12. 2024Applying RLAIF for Code Generation with API-usage in Lightweight LLMscited by 10
  13. 2023SteloCoder: a Decoder-Only LLM for Multi-Language to Python Code Translationcited by 9
  14. 2024Output Format Biases in the Evaluation of Large Language Models for Code Translationcited by 9
  15. 2023Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Modelscited by 6
  16. 2025The Code Barrier: What LLMs Actually Understand?cited by 5
  17. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  18. 2024A Study on Training and Developing Large Language Models for Behavior Tree Generationcited by 4
  19. 2025ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Groundingcited by 4
  20. 2025How Programming Concepts and Neurons Are Shared in Code Language Modelscited by 4
  21. 2025Function-to-Style Guidance of LLMs for Code Translationcited by 4
  22. 2023ZC3: Zero-Shot Cross-Language Code Clone Detectioncited by 3
  23. 2024Contextualized Data-Wrangling Code Generation in Computational Notebookscited by 3
  24. 2025Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Qualitycited by 3
  25. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  26. 2025RepoTransAgent: Multi-Agent LLM Framework for Repository-Aware Code Translationcited by 3
  27. 2025K2-V2: A 360-Open, Reasoning-Enhanced LLMcited by 3
  28. 2023Language Agnostic Code Embeddingscited by 2
  29. 2025Bridging Code Graphs and Large Language Models for Better Code Understandingcited by 2
  30. 2023SUT: Active Defects Probing for Transcompiler Modelscited by 1
  31. 2024Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuningcited by 1
  32. 2024Insights from Benchmarking Frontier Language Models on Web App Code Generationcited by 1
  33. 2025Collaboration is all you need: LLM Assisted Safe Code Translationcited by 1
  34. 2023CodeLL: A Lifelong Learning Dataset to Support the Co-Evolution of Data and Language Models of Codecited by 0
  35. 2024Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignmentscited by 0
  36. 2024On Training a Neural Network to Explain Binariescited by 0
  37. 2024In-Context Code-Text Learning for Bimodal Software Engineeringcited by 0
  38. 2025LoRACode: LoRA Adapters for Code Embeddingscited by 0

50 most cited of 54 All 54 (JSON)