mlsanityData to 14 Oct 2026

Calibrating Sequence likelihood Improves Conditional Language Generation

arXiv 2210.00045 ↗First posted 30 Sep 2022cs.CLcited by 121

Not on the map. Why?

Cited by 64 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Conditional language models are predominantly trained with maximum likelihood estimation (MLE), giving probability mass to sparsely observed target sequences. While MLE trained models assign high probability to plausible sequences given the context, the model probabilities often do not accurately rank-order generated sequences by quality. This has been empirically observed in beam search decoding as output quality degrading with large beam sizes, and decoding strategies benefiting from heuristics such as length normalization and repetition-blocking. In this work, we introduce sequence likelihood calibration (SLiC) where the likelihood of model generated sequences are calibrated to better align with reference sequences in the model's latent space. With SLiC, decoding heuristics become unnecessary and decoding candidates' quality significantly improves regardless of the decoding method. Furthermore, SLiC shows no sign of diminishing returns with model scale, and presents alternative ways to improve quality with limited training and inference budgets. With SLiC, we exceed or match SOTA results on a wide range of generation tasks spanning abstractive summarization, question generation, abstractive question answering and data-to-text generation, even with modest-sized models.

Key facts. Cited by 121 papers in the map (data through 14 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 30 Sep 2022.

Builds on

43 of 57 references.

References · 43

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  8. 2020Learning to summarize from human feedbackcited by 2,588
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  12. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
31 more
  1. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  2. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  3. 2015Teaching Machines to Read and Comprehendcited by 1,049
  4. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  5. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  6. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  7. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  8. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 365
  9. 2017Six Challenges for Neural Machine Translationcited by 335
  10. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  11. 2022UL2: Unifying Language Learning Paradigmscited by 295
  12. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  13. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  14. 2017Challenges in Data-to-Document Generationcited by 226
  15. 2020UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Trainingcited by 204
  16. 2017Learning to Ask: Neural Question Generation for Reading Comprehensioncited by 186
  17. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  18. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 157
  19. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 132
  20. 2018Abstractive Summarization of Reddit Posts with Multi-level Memory Networkscited by 102
  21. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  22. 2017Neural Question Generation from Text: A Preliminary Studycited by 78
  23. 2022SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarizationcited by 61
  24. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55
  25. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  26. 2020Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Modelscited by 44
  27. 2017Recurrent Neural Networks as Weighted Language Recognizerscited by 30
  28. 2018Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translationcited by 29
  29. 2019Multi-style Generative Reading Comprehensioncited by 14
  30. 2021Alleviating Exposure Bias via Contrastive Learning for Abstractive Text Summarizationcited by 14
  31. 2020Semantic Label Smoothing for Sequence to Sequence Problemscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 30; 2024: 53; 2025: 27; 2026: 9.

121 papers.

Sort:

By area:

  1. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
  2. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 401
  3. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
  4. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 291
  5. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
  6. 2023Large Language Model Alignment: A Surveycited by 243
  7. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 178
  8. 2023Controlled Decoding from Language Modelscited by 135
  9. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  10. 2024Theoretical guarantees on the best-of-n alignment policycited by 124
  11. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
  12. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
38 more
  1. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 121
  2. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 118
  3. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  4. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  5. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  6. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 95
  7. 2024Decoding-Time Language Model Alignment with Multiple Objectivescited by 94
  8. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  9. 2024Noise Contrastive Alignment of Language Models with Explicit Rewardscited by 85
  10. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  11. 2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 75
  12. 2024Multi-turn Reinforcement Learning from Preference Human Feedbackcited by 67
  13. 2024Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimizationcited by 67
  14. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  15. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  16. 2024Learn Your Reference Model for Real Good Alignmentcited by 57
  17. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  18. 2023Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Methodcited by 51
  19. 2023Improving Large Language Model Fine-tuning for Solving Math Problemscited by 49
  20. 2023Learning from models beyond fine-tuningcited by 48
  21. 2024Preference Learning Algorithms Do Not Learn Preference Rankingscited by 48
  22. 2024Asymptotics of Language Model Alignmentcited by 47
  23. 2023OmniVec: Learning robust representations with cross modal sharingcited by 44
  24. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  25. 2024LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversitycited by 42
  26. 2023Click: Controllable Text Generation with Sequence Likelihood Contrastive Learningcited by 41
  27. 2024Model Extrapolation Expedites Alignmentcited by 37
  28. 2024Improving the Robustness of Large Language Models via Consistency Alignmentcited by 36
  29. 2024InfAlign: Inference-aware language model alignmentcited by 35
  30. 2024Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Modelcited by 33
  31. 2023On Context Utilization in Summarization with Large Language Modelscited by 30
  32. 2023Aligning Language Models with Human Preferences via a Bayesian Approachcited by 27
  33. 2024Distributional Preference Alignment of LLMs via Optimal Transportcited by 26
  34. 2023On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Studycited by 25
  35. 2024Transforming and Combining Rewards for Aligning Large Language Modelscited by 25
  36. 2024Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Modelcited by 25
  37. 2024Information Theoretic Guarantees For Policy Alignment In Large Language Modelscited by 23
  38. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23

50 most cited of 121 All 121 (JSON)