# SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation

arXiv: https://arxiv.org/abs/2110.10774 · HTML: https://mlsanity.com/paper/2110.10774/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://mlsanity.com/paper/2004.07180/) (2020)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [On the Use of ArXiv as a Dataset](https://mlsanity.com/paper/1905.00075/) (2019)
- [Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles](https://mlsanity.com/paper/2010.14235/) (2020)
- [PaperRobot: Incremental Draft Generation of Scientific Ideas](https://mlsanity.com/paper/1905.07870/) (2019)
- [ParaSCI: A Large Scientific Paraphrase Dataset for Longer Paraphrase Generation](https://mlsanity.com/paper/2101.08382/) (2021)
- [Enhancing Scientific Papers Summarization with Citation Graph](https://mlsanity.com/paper/2104.03057/) (2021)
- [Learning to Reason for Text Generation from Scientific Tables](https://mlsanity.com/paper/2104.08296/) (2021)
- [Neural Academic Paper Generation](https://mlsanity.com/paper/1912.01982/) (2019)

## Built on by (most cited first, 22 of 22)

- [MAGE: Machine-generated Text Detection in the Wild](https://mlsanity.com/paper/2305.13242/) (2023)
- [DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning](https://mlsanity.com/paper/2410.20964/) (2024)
- [ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference](https://mlsanity.com/paper/2404.07947/) (2024)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network](https://mlsanity.com/paper/2303.14957/) (2023)
- [LitLLMs, LLMs for Literature Review: Are we there yet?](https://mlsanity.com/paper/2412.15249/) (2024)
- [SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generation](https://mlsanity.com/paper/2403.17768/) (2024)
- [MoSEs: Uncertainty-Aware AI-Generated Text Detection via Mixture of Stylistics Experts with Conditional Thresholds](https://mlsanity.com/paper/2509.02499/) (2025)
- [Diversity Boosts AI-Generated Text Detection](https://mlsanity.com/paper/2509.18880/) (2025)
- [Synthesis and Evaluation of a Domain-specific Large Data Set for Dungeons & Dragons](https://mlsanity.com/paper/2212.09080/) (2022)
- [Context-Enhanced Language Models for Generating Multi-Paper Citations](https://mlsanity.com/paper/2404.13865/) (2024)
- [KG-CTG: Citation Generation through Knowledge Graph-guided Large Language Models](https://mlsanity.com/paper/2404.09763/) (2024)
- [Synthetic Lyrics Detection Across Languages and Genres](https://mlsanity.com/paper/2406.15231/) (2024)
- [PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection](https://mlsanity.com/paper/2406.16288/) (2024)
- [Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection](https://mlsanity.com/paper/2501.03940/) (2025)
- [SciEvo: A 2 Million, 30-Year Cross-disciplinary Dataset for Temporal Scientometric Analysis](https://mlsanity.com/paper/2410.09510/) (2024)
- [RefuteBench 2.0 -- Agentic Benchmark for Dynamic Evaluation of LLM Responses to Refutation Instruction](https://mlsanity.com/paper/2502.18308/) (2025)
- [LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text](https://mlsanity.com/paper/2509.21269/) (2025)
- [Enhancing AI Research Paper Analysis: Methodology Component Extraction using Factored Transformer-based Sequence Modeling Approach](https://mlsanity.com/paper/2311.03401/) (2023)
- [SHADE: Semantic Hypernym Annotator for Domain-specific Entities -- DnD Domain Use Case](https://mlsanity.com/paper/2407.00407/) (2024)
- [AINL-Eval 2025 Shared Task: Detection of AI-Generated Scientific Abstracts in Russian](https://mlsanity.com/paper/2508.09622/) (2025)
- [READER: Reasoning-Enhanced AI-Generated Text Detection](https://mlsanity.com/paper/2605.25281/) (2026)

## Abstract (arXiv)

Generating texts in scientific papers requires not only capturing the content contained within the given input but also frequently acquiring the external information called textitcontext. We push forward the scientific text generation by proposing a new task, namely context-aware text generation in the scientific domain, aiming at exploiting the contributions of context in generated texts. To this end, we present a novel challenging large-scale Scientific Paper Dataset for ConteXt-Aware Text Generation (SciXGen), consisting of well-annotated 205,304 papers with full references to widely-used objects (e.g., tables, figures, algorithms) in a paper. We comprehensively benchmark, using state-of-the-arts, the efficacy of our newly constructed SciXGen dataset in generating description and paragraph. Our dataset and benchmarks will be made publicly available to hopefully facilitate the scientific text generation research.