mlsanityData to 7 Oct 2026

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X

arXiv 2303.17568 ↗First posted 30 Mar 2023cs.LG cs.AI cs.SEcited by 294

Not on the map. Why?

Cited by 32 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large pre-trained code generation models, such as OpenAI Codex, can generate syntax- and function-correct code, making the coding of programmers more productive and our pursuit of artificial general intelligence closer. In this paper, we introduce CodeGeeX, a multilingual model with 13 billion parameters for code generation. CodeGeeX is pre-trained on 850 billion tokens of 23 programming languages as of June 2022. Our extensive experiments suggest that CodeGeeX outperforms multilingual code models of similar scale for both the tasks of code generation and translation on HumanEval-X. Building upon HumanEval (Python only), we develop the HumanEval-X benchmark for evaluating multilingual models by hand-writing the solutions in C++, Java, JavaScript, and Go. In addition, we build CodeGeeX-based extensions on Visual Studio Code, JetBrains, and Cloud Studio, generating 4.7 billion tokens for tens of thousands of active users per week. Our user study demonstrates that CodeGeeX can help to increase coding efficiency for 83.4% of its users. Finally, CodeGeeX is publicly accessible and in Sep. 2022, we open-sourced its code, model weights (the version of 850B tokens), API, extensions, and HumanEval-X at https://github.com/THUDM/CodeGeeX.

Key facts. Cited by 294 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

37 of 52 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2016Layer Normalizationcited by 5,579
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
25 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  2. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  4. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  8. 2021Measuring Coding Challenge Competence With APPScited by 879
  9. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  11. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  12. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  13. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  14. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  15. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  16. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  17. 2019SPoC: Search-based Pseudocode to Codecited by 176
  18. 2021PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computationcited by 134
  19. 2022Productivity Assessment of Neural Code Completioncited by 97
  20. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  21. 2019TreeGen: A Tree-Based Transformer Architecture for Code Generationcited by 58
  22. 2022XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligencecited by 54
  23. 2021ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generationcited by 31
  24. 2021OneFlow: Redesign the Distributed Deep Learning Framework from Scratchcited by 15
  25. 2015On End-to-End Program Generation from User Intention by Deep Neural Networkscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 55; 2024: 110; 2025: 83; 2026: 46.

294 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  3. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  5. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  6. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  7. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  8. 2023Teaching Large Language Models to Self-Debugcited by 963
  9. 2023StarCoder: may the source be with you!cited by 790
  10. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  11. 2024A Survey on Large Language Models for Code Generationcited by 646
  12. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
38 more
  1. 2023Textbooks Are All You Needcited by 561
  2. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  3. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 346
  4. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  5. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  6. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  7. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  8. 2023Efficient Large Language Models: A Surveycited by 183
  9. 2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 166
  10. 2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 165
  11. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  12. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 126
  13. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  14. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 110
  15. 2024Large Language Models Meet NLP: A Surveycited by 102
  16. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  17. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  18. 2024HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalizationcited by 84
  19. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
  20. 2024AnalogCoder: Analog Circuit Design via Training-Free Code Generationcited by 82
  21. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedbackcited by 74
  22. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  23. 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 73
  24. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  25. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  26. 2024Getting the most out of your tokenizer for pre-training and domain adaptationcited by 65
  27. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 65
  28. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  29. 2024Understanding LLMs: A Comprehensive Overview from Training to Inferencecited by 58
  30. 2024Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Explorationcited by 58
  31. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  32. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  33. 2023ICE-Score: Instructing Large Language Models to Evaluate Codecited by 50
  34. 2023Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMscited by 45
  35. 2024LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generationcited by 45
  36. 2024Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedbackcited by 45
  37. 2024AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodologycited by 45
  38. 2025Parameter-Efficient Fine-Tuning for Foundation Modelscited by 43

50 most cited of 294 All 294 (JSON)