mlsanityData to 14 Oct 2026

CodeScholar: Growing Idiomatic Code Examples

arXiv 2312.15157 ↗First posted 23 Dec 2023cs.SE cs.LG cs.PLcited by 0

Not on the map. Why?

Abstract (arXiv)

Programmers often search for usage examples for API methods. A tool that could generate realistic, idiomatic, and contextual usage examples for one or more APIs would be immensely beneficial to developers. Such a tool would relieve the need for a deep understanding of the API landscape, augment existing documentation, and help discover interactions among APIs. We present CodeScholar, a tool that generates idiomatic code examples demonstrating the common usage of API methods. It includes a novel neural-guided search technique over graphs that grows the query APIs into idiomatic code examples. Our user study demonstrates that in 70% of cases, developers prefer CodeScholar generated examples over state-of-the-art large language models (LLM) like GPT3.5. We quantitatively evaluate 60 single and 25 multi-API queries from 6 popular Python libraries and show that across-the-board CodeScholar generates more realistic, diverse, and concise examples. In addition, we show that CodeScholar not only helps developers but also LLM-powered programming assistants generate correct code in a program synthesis setting.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 61 references that are in the map. First posted on arXiv 23 Dec 2023.

Builds on

21 of 61 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  10. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  11. 2021Measuring Coding Challenge Competence With APPScited by 902
  12. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
9 more
  1. 2022Execution-Based Evaluation for Open-Domain Code Generationcited by 91
  2. 2020Neural Subgraph Matchingcited by 47
  3. 2023Can Large Language Models Write Good Property-Based Tests?cited by 31
  4. 2018Aroma: Code Recommendation via Structural Code Searchcited by 29
  5. 2023Guiding Language Models of Code with Global Context using Monitorscited by 20
  6. 2014Mining Idioms from Source Codecited by 19
  7. 2022Code Search: A Survey of Techniques for Finding Codecited by 18
  8. 2015Learning API Usages from Bytecode: A Statistical Approachcited by 3
  9. 2020Exempla Gratis (E.G.): Code Examples for Freecited by 2

Built on by

Not cited yet.