mlsanityData to 7 Oct 2026

JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation

arXiv 1910.02216 ↗First posted 5 Oct 2019cs.LG cs.CL cs.SEcited by 49

Not on the map. Why?

Abstract (arXiv)

Interactive programming with interleaved code snippet cells and natural language markdown is recently gaining popularity in the form of Jupyter notebooks, which accelerate prototyping and collaboration. To study code generation conditioned on a long context history, we present JuICe, a corpus of 1.5 million examples with a curated test set of 3.7K instances based on online programming assignments. Compared with existing contextual code generation datasets, JuICe provides refined human-curated data, open-domain code, and an order of magnitude more training data. Using JuICe, we train models for two tasks: (1) generation of the API call sequence in a code cell, and (2) full code cell generation, both conditioned on the NL-Code history up to a particular code cell. Experiments using current baseline code generation models show that both context and distant supervision aid in generation, and that the dataset is challenging for current systems.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 14; 2023: 10; 2024: 14; 2025: 6; 2026: 1.

49 papers.

Sort:

By area:

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  3. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  4. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  5. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  6. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  7. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  8. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  9. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  10. 2022Execution-Based Evaluation for Open-Domain Code Generationcited by 89
  11. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 86
  12. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
37 more
  1. 2022Out of the BLEU: how should we assess quality of the Code Generation models?cited by 65
  2. 2023xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrievalcited by 46
  3. 2023Data Contamination Through the Lens of Timecited by 39
  4. 2022Training and Evaluating a Jupyter Notebook Data Science Assistantcited by 31
  5. 2022Execution-based Evaluation for Data Science Code Generation Modelscited by 31
  6. 2024Text2Cypher: Bridging Natural Language and Graph Databasescited by 30
  7. 2024Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarkingcited by 26
  8. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
  9. 2024Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Taskscited by 24
  10. 2024Unsupervised Evaluation of Code LLMs with Round-Trip Correctnesscited by 23
  11. 2023L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Modelscited by 22
  12. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  13. 2024Benchmarks and Metrics for Evaluations of Code Generation: A Critical Reviewcited by 19
  14. 2025FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluationcited by 17
  15. 2022Python Code Generation by Asking Clarification Questionscited by 16
  16. 2023Enhancing Large Language Models for Secure Code Generation: A Dataset-driven Study on Vulnerability Mitigationcited by 14
  17. 2025Software Development Life Cycle Perspective: A Survey of Benchmarks for Code Large Language Models and Agentscited by 11
  18. 2022Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generatorscited by 10
  19. 2025Towards an Understanding of Context Utilization in Code Intelligencecited by 10
  20. 2021Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generationcited by 8
  21. 2022CodePAD: Sequence-based Code Generation with Pushdown Automatoncited by 7
  22. 2021HAConvGNN: Hierarchical Attention Based Convolutional Graph Neural Network for Code Documentation Generation in Jupyter Notebookscited by 6
  23. 2024On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generationcited by 6
  24. 2024Data Analysis in the Era of Generative AIcited by 6
  25. 2024GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Modelscited by 6
  26. 2021Natural Language-Guided Programmingcited by 4
  27. 2021Lyra: A Benchmark for Turducken-Style Code Generationcited by 4
  28. 2023Capture the Flag: Uncovering Data Insights with Large Language Modelscited by 4
  29. 2022A Survey on Artificial Intelligence for Source Code: A Dialogue Systems Perspectivecited by 3
  30. 2022Code Generation for Unknown Libraries via Reading API Documentationscited by 3
  31. 2024Contextualized Data-Wrangling Code Generation in Computational Notebookscited by 3
  32. 2025Deep-Bench: Deep Learning Benchmark Dataset for Code Generationcited by 3
  33. 2025Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Qualitycited by 3
  34. 2025NoTeS-Bank: Benchmarking Neural Transcription and Search for Scientific Notes Understandingcited by 3
  35. 2023ADELT: Transpilation Between Deep Learning Frameworkscited by 2
  36. 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 2
  37. 2024Linguacodus: A Synergistic Framework for Transformative Code Generation in Machine Learning Pipelinescited by 0