mlsanityData to 14 Oct 2026

Zero-shot Conversational Summarization Evaluations with small Large Language Models

arXiv 2311.18041 ↗First posted 29 Nov 2023cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) exhibit powerful summarization abilities. However, their capabilities on conversational summarization remains under explored. In this work we evaluate LLMs (approx. 10 billion parameters) on conversational summarization and showcase their performance on various prompts. We show that the summaries generated by models depend on the instructions and the performance of LLMs vary with different instructions sometimes resulting steep drop in ROUGE scores if prompts are not selected carefully. We also evaluate the models with human evaluations and discuss the limitations of the models on conversational summarization

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 38 of its 54 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

38 of 54 references.

References · 38

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
26 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  5. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  7. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  8. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  9. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  10. 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
  11. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  12. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  13. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  14. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  15. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  16. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 302
  17. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  18. 2023Challenges and Applications of Large Language Modelscited by 254
  19. 2021DialogSum: A Real-Life Scenario Dialogue Summarization Datasetcited by 183
  20. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  21. 2019Answers Unite! Unsupervised Metrics for Reinforced Summarization Modelscited by 92
  22. 2023MeetingBank: A Benchmark Dataset for Meeting Summarizationcited by 84
  23. 2020Fill in the BLANC: Human-free quality estimation of document summariescited by 82
  24. 2023Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Methodcited by 76
  25. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  26. 2021GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generationcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026NC-Bench: An LLM Benchmark for Evaluating Conversational Competencecited by 1