mlsanityData to 7 Oct 2026

WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus

arXiv 2304.04358 ↗First posted 10 Apr 2023cs.CL cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce a new NLP task -- generating short factual articles with references for queries by mining supporting evidence from the Web. In this task, called WebBrain, the ultimate goal is to generate a fluent, informative, and factually-correct short article (e.g., a Wikipedia article) for a factual query unseen in Wikipedia. To enable experiments on WebBrain, we construct a large-scale dataset WebBrain-Raw by extracting English Wikipedia articles and their crawlable Wikipedia references. WebBrain-Raw is ten times larger than the previous biggest peer dataset, which can greatly benefit the research community. From WebBrain-Raw, we construct two task-specific datasets: WebBrain-R and WebBrain-G, which are used to train in-domain retriever and generator, respectively. Besides, we empirically analyze the performances of the current state-of-the-art NLP techniques on WebBrain and introduce a new framework ReGen, which enhances the generation factualness by improved evidence retrieval and task-specific pre-training for generation. Experiment results show that ReGen outperforms all baselines in both automatic and human evaluations.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 47 of its 65 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

47 of 65 references.

References · 47

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  12. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
35 more
  1. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  2. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  3. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  4. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  5. 2018Object Hallucination in Image Captioningcited by 661
  6. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  7. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  8. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  9. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  10. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  11. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  12. 2017Six Challenges for Neural Machine Translationcited by 334
  13. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  14. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  15. 2022Teaching language models to support answers with verified quotescited by 290
  16. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  17. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  18. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  19. 2015A Neural Conversational Modelcited by 195
  20. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  21. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  22. 2019Hierarchical Transformers for Multi-Document Summarizationcited by 132
  23. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  24. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  25. 2022Re2G: Retrieve, Rerank, Generatecited by 102
  26. 2018Retrieve and Refine: Improved Sequence Generation Models For Dialoguecited by 82
  27. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  28. 2017Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generationcited by 61
  29. 2021Neural Machine Translation with Monolingual Translation Memorycited by 61
  30. 2021Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representationscited by 60
  31. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  32. 2020Minimizing FLOPs to Learn Efficient Sparse Representationscited by 46
  33. 2021FRUIT: Faithfully Reflecting Updated Information in Textcited by 39
  34. 2019Generating Summaries with Topic Templates and Structured Convolutional Decoderscited by 20
  35. 2017Why We Read Wikipediacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 11; 2025: 7; 2026: 3.

29 papers.

Sort:

By area:

  1. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  2. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentationcited by 118
  3. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  4. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  5. 2023Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Taskscited by 73
  6. 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
  7. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  8. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  9. 2024Grounding Language Model with Chunking-Free In-Context Retrievalcited by 30
  10. 2024Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLPcited by 27
  11. 2024A Survey of Conversational Searchcited by 22
  12. 2023MegaWika: Millions of reports and their sources across 50 diverse languagescited by 9
17 more
  1. 2023MultiVENT: Multilingual Videos of Events with Aligned Natural Textcited by 7
  2. 2025WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipediacited by 6
  3. 2025Unbiased Reasoning for Knowledge-Intensive Tasks in Large Language Models via Conditional Front-Door Adjustmentcited by 6
  4. 2024L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding?cited by 5
  5. 2023LUNA: A Model-Based Universal Analysis Framework for Large Language Modelscited by 4
  6. 2025REVERSUM: A Multi-staged Retrieval-Augmented Generation Method to Enhance Wikipedia Tail Biographies through Personal Narrativescited by 3
  7. 2023Optimizing Factual Accuracy in Text Generation through Dynamic Knowledge Selectioncited by 2
  8. 2024WIKIGENBENCH: Exploring Full-length Wikipedia Generation under Real-World Scenariocited by 2
  9. 2024Think-then-Act: A Dual-Angle Evaluated Retrieval-Augmented Generationcited by 2
  10. 2024On the Capacity of Citation Generation by Large Language Modelscited by 2
  11. 2025Model-Document Protocol for AI Searchcited by 2
  12. 2025SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attentioncited by 2
  13. 2024Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Contextcited by 0
  14. 2025Hierarchical Memory Organization for Wikipedia Generationcited by 0
  15. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Surveycited by 0
  16. 2026Seeking Information with RAG-Assistants: Does Model Size Matter in Human-AI Collaborations?cited by 0
  17. 2026FrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computationcited by 0