# ScholarCopilot: Training Large Language Models for Academic Writing with Accurate Citations

arXiv: https://arxiv.org/abs/2504.00824 · HTML: https://mlsanity.com/paper/2504.00824/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 22 of 22)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Condenser: a Pre-training Architecture for Dense Retrieval](https://mlsanity.com/paper/2104.08253/) (2021)
- [The Semantic Scholar Open Data Platform](https://mlsanity.com/paper/2301.10140/) (2023)
- [OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs](https://mlsanity.com/paper/2411.14199/) (2024)
- [DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrievers](https://mlsanity.com/paper/2502.18460/) (2025)

## Built on by (most cited first, 10 of 10)

- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [SciFig: Towards Automating Editable Figure Generation for Scientific Papers](https://mlsanity.com/paper/2601.04390/) (2026)
- [Expert Preference-based Evaluation of Automated Related Work Generation](https://mlsanity.com/paper/2508.07955/) (2025)
- [RebuttalAgent: Strategic Persuasion in Academic Rebuttal via Theory of Mind](https://mlsanity.com/paper/2601.15715/) (2026)
- [Modelling and Classifying the Components of a Literature Review](https://mlsanity.com/paper/2508.04337/) (2025)
- [pAI/MSc: ML Theory Research with Humans on the Loop](https://mlsanity.com/paper/2604.20622/) (2026)
- [TIM: A Large-Scale Dataset and large Timeline Intelligence Model for Open-domain Timeline Summarization](https://mlsanity.com/paper/2506.21616/) (2025)
- [OUTLINEFORGE: Hierarchical Reinforcement Learning with Explicit States for Scientific Writing](https://mlsanity.com/paper/2601.09858/) (2026)
- [Paper Espresso: From Paper Overload to Research Insight](https://mlsanity.com/paper/2604.04562/) (2026)
- [ReCite: Agentic Reasoning for Faithful Citation](https://mlsanity.com/paper/2609.09156/) (2026)

## Abstract (arXiv)

Academic writing requires both coherent text generation and precise citation of relevant literature. Although recent Retrieval-Augmented Generation (RAG) systems have significantly improved factual accuracy in general-purpose text generation, their ability to support professional academic writing remains limited. In this work, we introduce ScholarCopilot, a unified framework designed to enhance existing large language models for generating professional academic articles with accurate and contextually relevant citations. ScholarCopilot dynamically determines when to retrieve scholarly references by generating a retrieval token [RET], which is then used to query a citation database. The retrieved references are fed into the model to augment the generation process. We jointly optimize both the generation and citation tasks within a single framework to improve efficiency. Our model is built upon Qwen-2.5-7B and trained on 500K papers from arXiv. It achieves a top-1 retrieval accuracy of 40.1% on our evaluation dataset, outperforming baselines such as E5-Mistral-7B-Instruct (15.0%) and BM25 (9.8%). On a dataset of 1,000 academic writing samples, ScholarCopilot scores 16.2/25 in generation quality -- measured across relevance, coherence, academic rigor, completeness, and innovation -- significantly surpassing all existing models, including much larger ones like the Retrieval-Augmented Qwen2.5-72B-Instruct. Human studies further demonstrate that ScholarCopilot, despite being a 7B model, significantly outperforms ChatGPT, achieving 100% preference in citation quality and over 70% in overall usefulness.