# RWGBench: Evaluating Scholarly Positioning in Related Work Generation

arXiv: https://arxiv.org/abs/2606.24894 · HTML: https://mlsanity.com/paper/2606.24894/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 70 references that are in the map.
- First posted on arXiv 30 May 2026.

## Builds on (most cited first, 30 of 33)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Hurdles to Progress in Long-form Question Answering](https://mlsanity.com/paper/2103.06332/) (2021)
- [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://mlsanity.com/paper/1904.01608/) (2019)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [Skill Retrieval Augmentation for Agentic AI](https://mlsanity.com/paper/2604.24594/) (2026)
- [Content-Based Citation Recommendation](https://mlsanity.com/paper/1802.08301/) (2018)
- [RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects](https://mlsanity.com/paper/2501.18365/) (2025)
- [Knowledge Editing through Chain-of-Thought](https://mlsanity.com/paper/2412.17727/) (2024)
- [JuDGE: Benchmarking Judgment Document Generation for Chinese Legal System](https://mlsanity.com/paper/2503.14258/) (2025)
- [CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding](https://mlsanity.com/paper/2105.10912/) (2021)
- [Related Work and Citation Text Generation: A Survey](https://mlsanity.com/paper/2404.11588/) (2024)
- [Mitigating Entity-Level Hallucination in Large Language Models](https://mlsanity.com/paper/2407.09417/) (2024)
- [SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation](https://mlsanity.com/paper/2508.15658/) (2025)
- [Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval](https://mlsanity.com/paper/2312.10661/) (2023)
- [Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models](https://mlsanity.com/paper/2506.04832/) (2025)
- [Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing](https://mlsanity.com/paper/2506.00536/) (2025)
- [CiteAssist: A System for Automated Preprint Citation and BibTeX Generation](https://mlsanity.com/paper/2407.03192/) (2024)

## Abstract (arXiv)

Large language models have shown strong fluency in scientific writing, yet the evaluation of related work generation (RWG) remains limited. Existing RWG evaluations largely inherit summarization-oriented metrics, using lexical or semantic similarity to reference sections as proxies for quality. However, related work writing is fundamentally a citation-level scholarly positioning task: it requires selecting, organizing, and framing prior work to clarify how a target paper relates to, differs from, and contributes beyond existing research.As a result, models may generate coherent and semantically-relevant text while exhibiting academically critical failures, such as inappropriate citation selection or misplaced references, that conventional metrics do not capture.To this end, we introduce RWGBench, a benchmark that evaluates RWG from the perspective of citation decision-making rather than text similarity. RWGBench is constructed from a large-scale collection of 40,108 computer science papers and a retrieval corpus of 1.09 million documents, with a carefully curated test set comprising 100 papers and their corresponding published related work sections.We propose a multi-dimensional evaluation framework that assesses citation selection, contextual appropriateness, organization, and discourse structure.Experiments reveal systematic limitations in current systems that are obscured by standard evaluations, while Oracle studies further disentangle retrieval-level and generation-level bottlenecks. Human evaluation further shows that our citation-centric metrics align substantially better with expert judgment than surface-level text metrics. RWGBench offers a citation-centric testbed for developing and evaluating related work generation systems that are better aligned with scholarly writing practices.