PyGen: A Collaborative Human-AI Approach to Python Package Creation
Not on the map. Why?
Abstract (arXiv)
The principles of automation and innovation serve as foundational elements for advancement in contemporary science and technology. Here, we introduce Pygen, an automation platform designed to empower researchers, technologists, and hobbyists to bring abstract ideas to life as core, usable software tools written in Python. Pygen leverages the immense power of autoregressive large language models to augment human creativity during the ideation, iteration, and innovation process. By combining state-of-the-art language models with open-source code generation technologies, Pygen has significantly reduced the manual overhead of tool development. From a user prompt, Pygen automatically generates Python packages for a complete workflow from concept to package generation and documentation. The findings of our work show that Pygen considerably enhances the researcher's productivity by enabling the creation of resilient, modular, and well-documented packages for various specialized purposes. We employ a prompt enhancement approach to distill the user's package description into increasingly specific and actionable. While being inherently an open-ended task, we have evaluated the generated packages and the documentation using Human Evaluation, LLM-based evaluation, and CodeBLEU, with detailed results in the results section. Furthermore, we documented our results, analyzed the limitations, and suggested strategies to alleviate them. Pygen is our vision of ethical automation, a framework that promotes inclusivity, accessibility, and collaborative development. This project marks the beginning of a large-scale effort towards creating tools where intelligent agents collaborate with humans to improve scientific and technological development substantially. Our code and generated examples are open-sourced at [https://github.com/GitsSaikat/Pygen]
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 37 of its 100 references that are in the map. First posted on arXiv 13 Nov 2024.
Builds on
37 of 100 references.
References · 37
- 2017Attention Is All You Needcited by 60,361
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2023Mistral 7Bcited by 3,164
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
25 more
- 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
- 2022Visual Prompt Tuningcited by 1,681
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
- 2019AutoML: A Survey of the State-of-the-Artcited by 390
- 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
- 2022Prompt-aligned Gradient for Prompt Tuningcited by 341
- 2023Large Language Models as Tool Makerscited by 303
- 2023Tool Learning with Foundation Modelscited by 302
- 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
- 2020Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learningcited by 174
- 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
- 2022Evolution through Large Modelscited by 134
- 2019An Open Source AutoML Benchmarkcited by 85
- 2020AutoML to Date and Beyond: Challenges and Opportunitiescited by 63
- 2024MASAI: Modular Architecture for Software-engineering AI Agentscited by 49
- 2023Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4cited by 48
- 2022Language Models as Inductive Reasonerscited by 32
- 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Reviewcited by 31
- 2021Information Directed Reward Learning for Reinforcement Learningcited by 24
- 2021"Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real Worldcited by 12
- 2023KAXAI: An Integrated Environment for Knowledge Analysis and Explainable AIcited by 4
- 2022Answer Fast: Accelerating BERT on the Tensor Streaming Processorcited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.