mlsanityData to 7 Oct 2026

From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape

arXiv 2312.10868 ↗First posted 18 Dec 2023cs.AI cs.CL cs.CYcited by 18

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

This comprehensive survey explored the evolving landscape of generative Artificial Intelligence (AI), with a specific focus on the transformative impacts of Mixture of Experts (MoE), multimodal learning, and the speculated advancements towards Artificial General Intelligence (AGI). It critically examined the current state and future trajectory of generative Artificial Intelligence (AI), exploring how innovations like Google's Gemini and the anticipated OpenAI Q* project are reshaping research priorities and applications across various domains, including an impact analysis on the generative AI research taxonomy. It assessed the computational challenges, scalability, and real-world implications of these technologies while highlighting their potential in driving significant progress in fields like healthcare, finance, and education. It also addressed the emerging academic challenges posed by the proliferation of both AI-themed and AI-generated preprints, examining their impact on the peer-review process and scholarly communication. The study highlighted the importance of incorporating ethical and human-centric methods in AI development, ensuring alignment with societal norms and welfare, and outlined a strategy for future AI research that focuses on a balanced and conscientious use of MoE, multimodality, and AGI in generative AI.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 153 of its 324 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

153 of 324 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Memory Augmented Language Models through Mixture of Word Experts15 Nov
  2. 2023Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer15 Oct
  3. 2023Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Models24 May
  4. 2023To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis22 May
  5. 2022A Mixture-of-Expert Approach to RL-based Dialogue Management31 May
  6. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Models17 Feb
  7. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Experts13 Dec

Other references · 146

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  10. 2021Ethical and social risks of harm from Language Modelscited by 957
  11. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  12. 2023BloombergGPT: A Large Language Model for Financecited by 923
38 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  3. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  4. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  5. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  6. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  9. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  10. 2022Efficiently Scaling Transformer Inferencecited by 529
  11. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  12. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  13. 2020Artificial Intelligence, Values and Alignmentcited by 443
  14. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  15. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  16. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
  17. 2022Multimodal Learning with Transformers: A Surveycited by 399
  18. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  19. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  20. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  21. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  22. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  23. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  24. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  25. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  26. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  27. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  28. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 199
  29. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  30. 2022Fine-tuning language models to find agreement among humans with diverse preferencescited by 189
  31. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  32. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  33. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  34. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  35. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  36. 2020Open Problems in Cooperative AIcited by 167
  37. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  38. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 156

50 most cited of 146 All 146 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 16; 2025: 0; 2026: 1.

18 papers.

Sort:

By area:

  1. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  2. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 32
  3. 2024Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoningcited by 24
  4. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Reviewcited by 19
  5. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  6. 2023Gemini Pro Defeated by GPT-4V: Evidence from Educationcited by 12
  7. 2024Parameter-Efficient Fine-Tuning of Large Language Models using Semantic Knowledge Tuningcited by 10
  8. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  9. 2024From COBIT to ISO 42001: Evaluating Cybersecurity Frameworks for Opportunities, Risks, and Regulatory Compliance in Commercializing Large Language Modelscited by 7
  10. 2024Octopus v3: Technical Report for On-device Sub-billion Multimodal AI Agentcited by 7
  11. 2024Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Surveycited by 6
  12. 2024Is Free Self-Alignment Possible?cited by 4
6 more
  1. 2024A Misleading Gallery of Fluid Motion by Generative Artificial Intelligencecited by 3
  2. 2024Over the Edge of Chaos? Excess Complexity as a Roadblock to Artificial General Intelligencecited by 2
  3. 2024GPT, Ontology, and CAABAC: A Tripartite Personalized Access Control Model Anchored by Compliance, Context and Attributecited by 1
  4. 2024Game of LLMs: Discovering Structural Constructs in Activities using Large Language Modelscited by 1
  5. 2024Embracing the Generative AI Revolution: Advancing Tertiary Education in Cybersecurity with GPTcited by 0
  6. 2026Accelerating AI-Powered Research: The PuppyChatter Framework for Usable and Flexible Toolingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrating Universal Generative AI Platforms in Educational Labs to Foster Critical Thinking and Digital Literacy
  2. 2023Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases
  3. 2023Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet
  4. 2024AI and Generative AI for Research Discovery and Summarization
  5. 2025ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquiry
  6. 2025AI4Research: A Survey of Artificial Intelligence for Scientific Research
  7. 2024Generative AI Systems: A Systems-based Perspective on Generative AI
  8. 2025ChatGPT and Gemini participated in the Korean College Scholastic Ability Test -- Earth Science I