mlsanityData to 14 Oct 2026

AcademicGPT: Empowering Academic Research

arXiv 2311.12315 ↗First posted 21 Nov 2023cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated exceptional capabilities across various natural language processing tasks. Yet, many of these advanced LLMs are tailored for broad, general-purpose applications. In this technical report, we introduce AcademicGPT, designed specifically to empower academic research. AcademicGPT is a continual training model derived from LLaMA2-70B. Our training corpus mainly consists of academic papers, thesis, content from some academic domain, high-quality Chinese data and others. While it may not be extensive in data scale, AcademicGPT marks our initial venture into a domain-specific GPT tailored for research area. We evaluate AcademicGPT on several established public benchmarks such as MMLU and CEval, as well as on some specialized academic benchmarks like PubMedQA, SCIEval, and our newly-created ComputerScienceQA, to demonstrate its ability from general knowledge ability, to Chinese ability, and to academic ability. Building upon AcademicGPT's foundation model, we also developed several applications catered to the academic area, including General Academic Question Answering, AI-assisted Paper Reading, Paper Review, and AI-assisted Title and Abstract Generation.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 67 of its 81 references that are in the map. First posted on arXiv 21 Nov 2023.

Builds on

67 of 81 references.

References · 67

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2016Layer Normalizationcited by 5,669
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
38 more
  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  8. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  13. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  14. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  15. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  16. 2020GLU Variants Improve Transformercited by 1,804
  17. 2019Root Mean Square Layer Normalizationcited by 1,480
  18. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  19. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  20. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  21. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  22. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  23. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  24. 2023BloombergGPT: A Large Language Model for Financecited by 939
  25. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  26. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  27. 2022Galactica: A Large Language Model for Sciencecited by 744
  28. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  29. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  30. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  31. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  32. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  33. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
  34. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  35. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 538
  36. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
  37. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  38. 2023Effective Long-Context Scaling of Foundation Modelscited by 303

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Self-Updatable Large Language Models by Integrating Context into Model Parameterscited by 21
  2. 2024Towards LifeSpan Cognitive Systemscited by 13
  3. 2025Unveiling the Merits and Defects of LLMs in Automatic Review Generation for Scientific Paperscited by 12
  4. 2024Automated Review Generation Method Based on Large Language Modelscited by 7
  5. 2025Bridging Social Psychology and LLM Reasoning: Conflict-Aware Meta-Review Generation via Cognitive Alignmentcited by 2
  6. 2026Breaking the Illusion of Review Reliability under Static Evaluation: SCOPE Fuzzing for LLM-based Scientific Reviewerscited by 0