mlsanityData to 7 Oct 2026

SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization

arXiv 2106.01890 ↗First posted 3 Jun 2021cs.CLcited by 131

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i.e., quality estimation) assisted by contrastive learning. Experimental results show that, with minor modification over existing top-scoring systems, SimCLS can improve the performance of existing top-performing models by a large margin. Particularly, 2.51 absolute improvement against BART and 2.50 over PEGASUS w.r.t ROUGE-1 on the CNN/DailyMail dataset, driving the state-of-the-art performance to a new level. We have open-sourced our codes and results: https://github.com/yixinL7/SimCLS. Results of our proposed models have been deployed into ExplainaBoard platform, which allows researchers to understand our systems in a more fine-grained way.

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Builds on 32 of its 39 references that are in the map. First posted on arXiv 3 Jun 2021.

Builds on

32 of 39 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  10. 2015Teaching Machines to Read and Comprehendcited by 1,039
  11. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  12. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
20 more
  1. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  2. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 415
  3. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  4. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  5. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 221
  6. 2020Extractive Summarization as Text Matchingcited by 189
  7. 2020GSum: A General Framework for Guided Neural Abstractive Summarizationcited by 129
  8. 2016Sequence-to-Sequence Learning as Beam-Search Optimizationcited by 126
  9. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  10. 2020SciREX: A Challenge Dataset for Document-Level Information Extractioncited by 111
  11. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
  12. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  13. 2016Reward Augmented Maximum Likelihood for Neural Structured Predictioncited by 80
  14. 2020Contrastive Learning with Adversarial Perturbations for Conditional Text Generationcited by 56
  15. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55
  16. 2021ExplainaBoard: An Explainable Leaderboard for NLPcited by 50
  17. 2021RefSum: Refactoring Neural Summarizationcited by 25
  18. 2019Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarizationcited by 24
  19. 2020Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learningcited by 17
  20. 2020KoBE: Knowledge-Based Machine Translation Evaluationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 46; 2023: 42; 2024: 18; 2025: 10; 2026: 5.

131 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  3. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  4. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  5. 2022A Contrastive Framework for Neural Text Generationcited by 224
  6. 2023Large Language Model Routing with Benchmark Datasetscited by 166
  7. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  8. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 141
  9. 2022Calibrating Sequence likelihood Improves Conditional Language Generationcited by 121
  10. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  11. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  12. 2022Language Models Can See: Plugging Visual Controls in Text Generationcited by 89
38 more
  1. 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
  2. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  3. 2022SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarizationcited by 61
  4. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  5. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  6. 2021A Survey on Dialogue Summarization: Recent Advances and New Frontierscited by 51
  7. 2022FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarizationcited by 49
  8. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  9. 2024Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routingcited by 43
  10. 2022Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learningcited by 41
  11. 2022Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metricscited by 37
  12. 2024Leveraging Large Language Models for NLG Evaluation: Advances and Challengescited by 37
  13. 2024Routoo: Learning to Route to Large Language Models Effectivelycited by 34
  14. 2021TaCL: Improving BERT Pre-training with Token-aware Contrastive Learningcited by 32
  15. 2021Sequence Level Contrastive Learning for Text Summarizationcited by 31
  16. 2023Faithfulness-Aware Decoding Strategies for Abstractive Summarizationcited by 31
  17. 2022Towards Improving Faithfulness in Abstractive Summarizationcited by 27
  18. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  19. 2023Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluationcited by 24
  20. 2023Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generationcited by 24
  21. 2022CoNT: Contrastive Neural Text Generationcited by 21
  22. 2021From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compressioncited by 18
  23. 2022Improving Word Translation via Two-Stage Contrastive Learningcited by 18
  24. 2022Language modeling via stochastic processescited by 18
  25. 2023AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrievalcited by 18
  26. 2023Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarizationcited by 17
  27. 2023Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answeringcited by 15
  28. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  29. 2022Salience Allocation as Guidance for Abstractive Summarizationcited by 14
  30. 2023Summaries as Captions: Generating Figure Captions for Scientific Documents with Automated Text Summarizationcited by 14
  31. 2024PickLLM: Context-Aware RL-Assisted Large Language Model Routingcited by 13
  32. 2022Neural Machine Translation with Contrastive Translation Memoriescited by 12
  33. 2022Towards Better Chinese-centric Neural Machine Translation for Low-resource Languagescited by 11
  34. 2022A Token-level Contrastive Framework for Sign Language Translationcited by 10
  35. 2023Lightweight reranking for language model generationscited by 10
  36. 2023Large-Scale and Multi-Perspective Opinion Summarization with Diverse Review Subsetscited by 10
  37. 2024TriSum: Learning Summarization Ability from Large Language Models with Structured Rationalecited by 10
  38. 2021CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarizationcited by 9

50 most cited of 131 All 131 (JSON)