mlsanityData to 7 Oct 2026

Generating Wikipedia by Summarizing Long Sequences

arXiv 1801.10198 ↗First posted 30 Jan 2018cs.CLcited by 333

Not on the map. Why?

Abstract (arXiv)

We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstractive model to generate the article. For the abstractive model, we introduce a decoder-only architecture that can scalably attend to very long sequences, much longer than typical encoder- decoder architectures used in sequence transduction. We show that this model can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles. When given reference documents, we show it can extract relevant factual information as reflected in perplexity, ROUGE scores and human evaluations.

Key facts. Cited by 333 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 78; 2022: 70; 2023: 73; 2024: 53; 2025: 42; 2026: 16.

333 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  6. 2023A decoder-only foundation model for time-series forecastingcited by 650
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2021A Survey of Transformerscited by 399
  9. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  10. 2021Random Feature Attentioncited by 350
  11. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  12. 2021Efficient Attentions for Long Document Summarizationcited by 342
38 more
  1. 2022UL2: Unifying Language Learning Paradigmscited by 292
  2. 2021Recursively Summarizing Books with Human Feedbackcited by 287
  3. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  4. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  5. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  6. 2023ChatGPT: Jack of all trades, master of nonecited by 204
  7. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  8. 2023Small-scale proxies for large-scale Transformer training instabilitiescited by 182
  9. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  10. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  11. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  12. 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 149
  13. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  14. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127
  15. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  16. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 115
  17. 2023Retrieval meets Long Context Large Language Modelscited by 114
  18. 2022Training Language Models with Memory Augmentationcited by 112
  19. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  20. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 103
  21. 2021Luna: Linear Unified Nested Attentioncited by 96
  22. 2021Finetuning Pretrained Transformers into RNNscited by 94
  23. 2021Learning to Match Features with Seeded Graph Matching Networkcited by 92
  24. 2022What Language Model to Train if You Have One Million GPU Hours?cited by 92
  25. 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 84
  26. 2021MS2: Multi-Document Summarization of Medical Studiescited by 80
  27. 2021SummScreen: A Dataset for Abstractive Screenplay Summarizationcited by 80
  28. 2022Personalized Prompt Learning for Explainable Recommendationcited by 79
  29. 2023MatFormer: Nested Transformer for Elastic Inferencecited by 79
  30. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  31. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 78
  32. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Modelscited by 78
  33. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  34. 2023Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarizationcited by 70
  35. 2021Token Pooling in Vision Transformerscited by 68
  36. 2021KVT: k-NN Attention for Boosting Vision Transformerscited by 64
  37. 2022Single Stage Virtual Try-on via Deformable Attention Flowscited by 64
  38. 2021PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarizationcited by 62

50 most cited of 333 All 333 (JSON)