mlsanityData to 7 Oct 2026

OpenAGI: When LLM Meets Domain Experts

arXiv 2304.04370 ↗First posted 10 Apr 2023cs.AI cs.CL cs.LGcited by 196

Not on the map. Why?

Cited by 55 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Human Intelligence (HI) excels at combining basic skills to solve complex tasks. This capability is vital for Artificial Intelligence (AI) and should be embedded in comprehensive AI Agents, enabling them to harness expert models for complex task-solving towards Artificial General Intelligence (AGI). Large Language Models (LLMs) show promising learning and reasoning abilities, and can effectively use external models, tools, plugins, or APIs to tackle complex problems. In this work, we introduce OpenAGI, an open-source AGI research and development platform designed for solving multi-step, real-world tasks. Specifically, OpenAGI uses a dual strategy, integrating standard benchmark tasks for benchmarking and evaluation, and open-ended tasks including more expandable models, tools, plugins, or APIs for creative problem-solving. Tasks are presented as natural language queries to the LLM, which then selects and executes appropriate models. We also propose a Reinforcement Learning from Task Feedback (RLTF) mechanism that uses task results to improve the LLM's task-solving ability, which creates a self-improving AI feedback loop. While we acknowledge that AGI is a broad and multifaceted research challenge with no singularly defined solution path, the integration of LLMs with domain-specific expert models, inspired by mirroring the blend of general and specialized intelligence in humans, offers a promising approach towards AGI. We are open-sourcing the OpenAGI project's code, dataset, benchmarks, evaluation methods, and the UI demo to foster community involvement in AGI advancement: https://github.com/agiresearch/OpenAGI.

Key facts. Cited by 196 papers in the map (data through 7 Oct 2026). Builds on 51 of its 64 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

51 of 64 references.

References · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2017Proximal Policy Optimization Algorithmscited by 14,441
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  12. 2020End-to-End Object Detection with Transformerscited by 7,384
38 more
  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  13. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  14. 2019Towards VQA Models That Can Readcited by 1,696
  15. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  16. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  17. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  18. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  19. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  20. 2022Red Teaming Language Models with Language Modelscited by 983
  21. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  22. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  23. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  24. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  25. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  26. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  27. 2023Augmented Language Models: a Surveycited by 398
  28. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 396
  29. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  30. 2019FinBERT: Financial Sentiment Analysis with Pre-trained Language Modelscited by 378
  31. 2020Autoregressive Entity Retrievalcited by 333
  32. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  33. 2017Real-Time User-Guided Image Colorization with Learned Deep Priorscited by 123
  34. 2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
  35. 2022Language Models are General-Purpose Interfacescited by 96
  36. 2022Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restorationcited by 72
  37. 2017Thinking Fast and Slow: Optimization Decomposition Across Timescalescited by 60
  38. 2022Learn Basic Skills and Reuse: Modularized Adaptive Neural Architecture Search (MANAS)cited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 42; 2024: 85; 2025: 54; 2026: 15.

196 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  5. 2023A Comprehensive Overview of Large Language Modelscited by 639
  6. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  7. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  8. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  9. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
  10. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  11. 2024Tool Learning with Large Language Models: A Surveycited by 221
  12. 2023Language is All a Graph Needscited by 219
38 more
  1. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 166
  2. 2023War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Warscited by 143
  3. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
  4. 2024CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluationcited by 139
  5. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  6. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 102
  7. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  8. 2023AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learncited by 97
  9. 2024AIOS: LLM Agent Operating Systemcited by 94
  10. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  11. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 88
  12. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  13. 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 80
  14. 2023NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classescited by 78
  15. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  16. 2024AutoFlow: Automated Workflow Generation for Large Language Model Agentscited by 76
  17. 2023Large Language Models for Generative Recommendation: A Survey and Visionary Discussionscited by 72
  18. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  19. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  20. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  21. 2024TrustAgent: Towards Safe and Trustworthy LLM-based Agentscited by 62
  22. 2024When Search Engine Services meet Large Language Models: Visions and Challengescited by 59
  23. 2024MMInA: Benchmarking Multihop Multimodal Internet Agentscited by 55
  24. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 55
  25. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  26. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usagecited by 54
  27. 2024Visual Agents as Fast and Slow Thinkerscited by 53
  28. 2023Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaborationcited by 52
  29. 2024TDAG: A Multi-Agent Framework based on Dynamic Task Decomposition and Agent Generationcited by 50
  30. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  31. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  32. 2023VIP5: Towards Multimodal Foundation Models for Recommendationcited by 45
  33. 2024Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Modelscited by 45
  34. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 44
  35. 2023A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verificationcited by 42
  36. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 42
  37. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  38. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agentscited by 39

50 most cited of 196 All 196 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub
  2. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
  3. 2025OpenSIR: Open-Ended Self-Improving Reasoner
  4. 2024Distilling LLMs' Decomposition Abilities into Compact Language Models
  5. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Model
  6. 2024Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
  7. 2025Challenges and Applications of Large Language Models: A Comparison of GPT and DeepSeek family of models
  8. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs