Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges
Not on the map. Why?
Abstract (arXiv)
Text generation has become more accessible than ever, and the growing interest in these systems, especially those using large language models, has spurred a surge in related publications. We provide a systematic literature review comprising 257 papers, covering the period from January 2017 to December 2025. This review categorizes text generation contributions into five main tasks: open-ended text generation, summarization, translation, paraphrasing, and question answering. For each task in our taxonomy, we review relevant characteristics and key subtasks. We assess current approaches for evaluating text generation systems, covering model-free, model-based, and human evaluation. Our investigation shows several task-specific challenges (e.g., missing datasets for multi-document summarization, lack of coherence in story generation, and difficulties in complex reasoning for question answering). We further discuss nine challenges common to all tasks and sub-tasks in recent text generation papers: bias, reasoning, hallucinations, misuse, privacy, interpretability, transparency, datasets, and computing. This systematic literature review targets two main audiences: early-career researchers in natural language processing seeking an overview of the field and promising research directions, and senior researchers who need a recent overview of the main tasks, evaluation, challenges, and mitigation strategies.
Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 180 of its 292 references that are in the map. First posted on arXiv 24 May 2024.
Builds on
180 of 292 references.
References · 180
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
38 more
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
- 2023A Survey of Large Language Modelscited by 2,576
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2018Hierarchical Neural Story Generationcited by 1,193
- 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
- 2018Model Cards for Model Reportingcited by 1,080
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
- 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
- 2020Efficient Transformers: A Surveycited by 790
- 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
- 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
- 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
- 2023Bias and Fairness in Large Language Models: A Surveycited by 639
- 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
- 2019ELI5: Long Form Question Answeringcited by 575
- 2019Defending Against Neural Fake Newscited by 559
- 2019Text Summarization with Pretrained Encoderscited by 556
- 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
- 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
50 most cited of 180 All 180 (JSON)
Built on by
Citing papers per year: 2024: 4; 2025: 11; 2026: 3.
18 papers.
Sort:
By area:
- 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 29
- 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
- 2025Stay Focused: Problem Drift in Multi-Agent Debatecited by 23
- 2025MALLM: Multi-Agent Large Language Models Frameworkcited by 15
- 2024Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Frameworkcited by 9
- 2025Statistical Multicriteria Evaluation of LLM-Generated Textcited by 9
- 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 9
- 2025On the Empirical Power of Goodness-of-Fit Tests in Watermark Detectioncited by 5
- 2024SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detectioncited by 4
- 2026Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generationcited by 4
- 2024Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinementcited by 3
- 2025Unleashing Scientific Reasoning for Bio-experimental Protocol Generation via Structured Component-based Reward Mechanismcited by 3
6 more
- 2025CachePrune: Teaching LLMs What Not to Follow via KV-Cache Editingcited by 2
- 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokenscited by 2
- 2025From Text to Graph: Leveraging Graph Neural Networks for Enhanced Explainability in NLPcited by 1
- 2026Are complicated loss functions necessary for teaching LLMs to reason?cited by 1
- 2025Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answeringcited by 0
- 2026From Intention to Text: AI-Supported Goal Setting in Academic Writingcited by 0
- 2026From Intention to Text: AI-Supported Goal Setting in Academic Writing17 Apr 2026
- 2026Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generation12 Apr 2026
- 2026Are complicated loss functions necessary for teaching LLMs to reason?19 Mar 2026
- 2025Unleashing Scientific Reasoning for Bio-experimental Protocol Generation via Structured Component-based Reward Mechanism17 Oct 2025
- 2025On the Empirical Power of Goodness-of-Fit Tests in Watermark Detection4 Oct 2025
- 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokens24 Sep 2025
- 2025MALLM: Multi-Agent Large Language Models Framework15 Sep 2025
- 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM7 Aug 2025
- 2025Statistical Multicriteria Evaluation of LLM-Generated Text22 Jun 2025
- 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models22 May 2025
- 2025Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering5 May 2025
- 2025CachePrune: Teaching LLMs What Not to Follow via KV-Cache Editing29 Apr 2025
6 more
- 2025From Text to Graph: Leveraging Graph Neural Networks for Enhanced Explainability in NLP2 Apr 2025
- 2025Stay Focused: Problem Drift in Multi-Agent Debate26 Feb 2025
- 2024Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement30 Dec 2024
- 2024Multi-Agent Large Language Models for Conversational Task-Solving30 Oct 2024
- 2024Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Framework24 Oct 2024
- 2024SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection22 Aug 2024
- 2025MALLM: Multi-Agent Large Language Models Frameworkcited by 15
- 2024Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinementcited by 3
- 2025Unleashing Scientific Reasoning for Bio-experimental Protocol Generation via Structured Component-based Reward Mechanismcited by 3
- 2026Are complicated loss functions necessary for teaching LLMs to reason?cited by 1
- 2026Are complicated loss functions necessary for teaching LLMs to reason?19 Mar 2026
- 2025Unleashing Scientific Reasoning for Bio-experimental Protocol Generation via Structured Component-based Reward Mechanism17 Oct 2025
- 2025MALLM: Multi-Agent Large Language Models Framework15 Sep 2025
- 2024Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement30 Dec 2024
- 2024Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Frameworkcited by 9
- 2025Statistical Multicriteria Evaluation of LLM-Generated Textcited by 9
- 2025On the Empirical Power of Goodness-of-Fit Tests in Watermark Detectioncited by 5
- 2024SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detectioncited by 4
- 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokenscited by 2
- 2025From Text to Graph: Leveraging Graph Neural Networks for Enhanced Explainability in NLPcited by 1
- 2025Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answeringcited by 0
- 2026From Intention to Text: AI-Supported Goal Setting in Academic Writingcited by 0
- 2026From Intention to Text: AI-Supported Goal Setting in Academic Writing17 Apr 2026
- 2025On the Empirical Power of Goodness-of-Fit Tests in Watermark Detection4 Oct 2025
- 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokens24 Sep 2025
- 2025Statistical Multicriteria Evaluation of LLM-Generated Text22 Jun 2025
- 2025Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering5 May 2025
- 2025From Text to Graph: Leveraging Graph Neural Networks for Enhanced Explainability in NLP2 Apr 2025
- 2024Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Framework24 Oct 2024
- 2024SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection22 Aug 2024