mlsanityData to 7 Oct 2026

Towards Building Specialized Generalist AI with System 1 and System 2 Fusion

arXiv 2407.08642 ↗First posted 11 Jul 2024cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

In this perspective paper, we introduce the concept of Specialized Generalist Artificial Intelligence (SGAI or simply SGI) as a crucial milestone toward Artificial General Intelligence (AGI). Compared to directly scaling general abilities, SGI is defined as AI that specializes in at least one task, surpassing human experts, while also retaining general abilities. This fusion path enables SGI to rapidly achieve high-value areas. We categorize SGI into three stages based on the level of mastery over professional skills and generality performance. Additionally, we discuss the necessity of SGI in addressing issues associated with large language models, such as their insufficient generality, specialized capabilities, uncertainty in innovation, and practical applications. Furthermore, we propose a conceptual framework for developing SGI that integrates the strengths of Systems 1 and 2 cognitive processing. This framework comprises three layers and four key components, which focus on enhancing individual abilities and facilitating collaborative evolution. We conclude by summarizing the potential challenges and suggesting future directions. We hope that the proposed SGI will provide insights into further research and applications towards achieving AGI.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 166 of its 234 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

166 of 234 references.

References · 166

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2017Proximal Policy Optimization Algorithmscited by 14,441
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  10. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  12. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
38 more
  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  10. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  12. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  13. 2023Let's Verify Step by Stepcited by 3,598
  14. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  15. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  16. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  17. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  18. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  19. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  20. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  21. 2018Deep contextualized word representationscited by 2,285
  22. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  23. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  24. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  25. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  26. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  27. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  28. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  29. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  30. 2023Teaching Large Language Models to Self-Debugcited by 963
  31. 2021Ethical and social risks of harm from Language Modelscited by 957
  32. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  33. 2022In-context Learning and Induction Headscited by 930
  34. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  35. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  36. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  37. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  38. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752

50 most cited of 166 All 166 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 6.

8 papers.

  1. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  2. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 52
  3. 2024InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactionscited by 48
  4. 2024AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistantcited by 27
  5. 2025Exploiting Edited Large Language Models as General Scientific Optimizerscited by 7
  6. 2025OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?cited by 6
  7. 2025Making, not Taking, the Best of Ncited by 4
  8. 2025Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanismcited by 2