mlsanityData to 7 Oct 2026

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face

arXiv 2303.17580 ↗First posted 30 Mar 2023cs.CL cs.AI cs.CVcited by 1,173

Not on the map. Why?

Cited by 512 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Solving complicated AI tasks with different domains and modalities is a key step toward artificial general intelligence. While there are numerous AI models available for various domains and modalities, they cannot handle complicated AI tasks autonomously. Considering large language models (LLMs) have exhibited exceptional abilities in language understanding, generation, interaction, and reasoning, we advocate that LLMs could act as a controller to manage existing AI models to solve complicated AI tasks, with language serving as a generic interface to empower this. Based on this philosophy, we present HuggingGPT, an LLM-powered agent that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., Hugging Face) to solve AI tasks. Specifically, we use ChatGPT to conduct task planning when receiving a user request, select models according to their function descriptions available in Hugging Face, execute each subtask with the selected AI model, and summarize the response according to the execution results. By leveraging the strong language capability of ChatGPT and abundant AI models in Hugging Face, HuggingGPT can tackle a wide range of sophisticated AI tasks spanning different modalities and domains and achieve impressive results in language, vision, speech, and other challenging tasks, which paves a new way towards the realization of artificial general intelligence.

Key facts. Cited by 1,173 papers in the map (data through 7 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

32 of 44 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
20 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  10. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  11. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  12. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  13. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  14. 2022PAL: Program-aided Language Modelscited by 635
  15. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  16. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  17. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  18. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  19. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  20. 2023ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptionscited by 86

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 290; 2024: 281; 2025: 288; 2026: 314.

1,173 papers.

Sort:

By area:

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  6. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  8. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  11. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  12. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
38 more
  1. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  2. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  5. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  6. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  9. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  10. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  12. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  13. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  14. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  15. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  16. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  17. 2024Large Language Models: A Surveycited by 506
  18. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  19. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  20. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  21. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
  22. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  23. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  24. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  25. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  26. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  27. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
  28. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  29. 2023Tool Learning with Foundation Modelscited by 302
  30. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  31. 2023Large Language Models as Tool Makerscited by 293
  32. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  33. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  34. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
  35. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  36. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  37. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  38. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273

50 most cited of 1,173 All 1,173 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ChatGPT Alternative Solutions: Large Language Models Survey
  2. 2025A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks
  3. 2023ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?
  4. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
  5. 2025HuggingR4: A Progressive Reasoning Framework for Discovering Optimal Model Companions
  6. 2025Challenges and Applications of Large Language Models: A Comparison of GPT and DeepSeek family of models
  7. 2024CHATATC: Large Language Model-Driven Conversational Agents for Supporting Strategic Air Traffic Flow Management
  8. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey