mlsanityData to 14 Oct 2026

AixBench: A Code Generation Benchmark Dataset

arXiv 2206.13179 ↗First posted 27 Jun 2022cs.SEcited by 22

Not on the map. Why?

Abstract (arXiv)

We present a benchmark dataset for evaluating method-level code generation task. The benchmark contains a dataset of 175 samples for automated evaluation and a dataset of 161 samples for manual evaluation. We also present a new metric for automatically evaluating the correctness of the generated code, and a set of criteria to manually evaluating the overall quality of the generated code.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Jun 2022.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 6; 2025: 8; 2026: 1.

22 papers.

Sort:

By area:

  1. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 138
  2. 2024EffiBench: Benchmarking the Efficiency of Automatically Generated Codecited by 64
  3. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  4. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  5. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 57
  6. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
  7. 2025LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarkscited by 37
  8. 2024Automatic Programming: Large Language Models and Beyondcited by 36
  9. 2024mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generationcited by 32
  10. 2025ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generationcited by 26
  11. 2023AceCoder: Utilizing Existing Code to Enhance Code Generationcited by 25
  12. 2025A Study of LLMs' Preferences for Libraries and Programming Languagescited by 20
10 more
  1. 2024PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)cited by 19
  2. 2023FRANC: A Lightweight Framework for High-Quality Code Generationcited by 14
  3. 2024Top General Performance = Top Domain Performance? DomainCodeBench: A Multi-domain Code Generation Benchmarkcited by 13
  4. 2025Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Qualitycited by 4
  5. 2022A Survey on Natural Language Processing for Programmingcited by 3
  6. 2025Deep-Bench: Deep Learning Benchmark Dataset for Code Generationcited by 3
  7. 2025Mechanistic Understanding of Language Models in Syntactic Code Completioncited by 2
  8. 2025CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluationcited by 1
  9. 2024Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programmingcited by 0
  10. 2026Improving LLM Code Generation via Requirement-Aware Curriculum Reinforcement Learningcited by 0