mlsanityData to 14 Oct 2026

Challenges in Domain-Specific Abstractive Summarization and How to Overcome them

arXiv 2307.00963 ↗First posted 3 Jul 2023cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Large Language Models work quite well with general-purpose data and many tasks in Natural Language Processing. However, they show several limitations when used for a task such as domain-specific abstractive text summarization. This paper identifies three of those limitations as research problems in the context of abstractive text summarization: 1) Quadratic complexity of transformer-based models with respect to the input text length; 2) Model Hallucination, which is a model's ability to generate factually incorrect text; and 3) Domain Shift, which happens when the distribution of the model's training and test corpus is not the same. Along with a discussion of the open research questions, this paper also provides an assessment of existing state-of-the-art techniques relevant to domain-specific text summarization to address the research gaps.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

29 of 40 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  8. 2020Reformer: The Efficient Transformercited by 1,668
  9. 2020Rethinking Attention with Performerscited by 1,601
  10. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  11. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  12. 2020Efficient Transformers: A Surveycited by 790
17 more
  1. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  2. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 217
  3. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 211
  4. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
  5. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 152
  6. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  7. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  8. 2019Assessing The Factual Accuracy of Generated Textcited by 103
  9. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  10. 2020GO FIGURE: A Meta Evaluation of Factuality in Summarizationcited by 76
  11. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 76
  12. 2017Text Summarization Techniques: A Brief Surveycited by 59
  13. 2021Automated Fact-Checking: A Surveycited by 55
  14. 2021Efficient Domain Adaptation of Language Models via Adaptive Tokenizationcited by 34
  15. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  16. 2020Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarizationcited by 13
  17. 2019Joint Parsing and Generation for Abstractive Summarizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 1.

5 papers.

  1. 2023Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AIcited by 33
  2. 2024WisPerMed at BioLaySumm: Adapting Autoregressive Large Language Models for Lay Summarization of Scientific Articlescited by 1
  3. 2024Enhancing Answer Attribution for Faithful Text Generation with Large Language Modelscited by 0
  4. 2025AugAbEx: Bridging Abstractive and Extractive Legal Summarizationcited by 0
  5. 2026Hallucination or Creativity: How to Evaluate AI-Generated Scientific Stories?cited by 0