# Large Margin Neural Language Model

arXiv: https://arxiv.org/abs/1808.08987 · HTML: https://mlsanity.com/paper/1808.08987/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2018.

## Built on by (most cited first, 12 of 12)

- [Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation](https://mlsanity.com/paper/2109.12242/) (2021)
- [Personalized Showcases: Generating Multi-Modal Explanations for Recommendations](https://mlsanity.com/paper/2207.00422/) (2022)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [ASR Rescoring and Confidence Estimation with ELECTRA](https://mlsanity.com/paper/2110.01857/) (2021)
- [Increasing Visual Awareness in Multimodal Neural Machine Translation from an Information Theoretic Perspective](https://mlsanity.com/paper/2210.08478/) (2022)
- [Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Information](https://mlsanity.com/paper/2109.05712/) (2021)
- [A Survey on Multi-modal Machine Translation: Tasks, Methods and Challenges](https://mlsanity.com/paper/2405.12669/) (2024)
- [C3: Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues](https://mlsanity.com/paper/2106.08914/) (2021)
- [FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?](https://mlsanity.com/paper/2307.04114/) (2023)
- [From Construction to Injection: Edit-Based Fingerprints for Large Language Models](https://mlsanity.com/paper/2509.03122/) (2025)
- [Learning to Write with Coherence From Negative Examples](https://mlsanity.com/paper/2209.10922/) (2022)
- [Harnessing non-adversarial robustness in large language models](https://mlsanity.com/paper/2605.29816/) (2026)

## Abstract (arXiv)

We propose a large margin criterion for training neural language models. Conventionally, neural language models are trained by minimizing perplexity (PPL) on grammatical sentences. However, we demonstrate that PPL may not be the best metric to optimize in some tasks, and further propose a large margin formulation. The proposed method aims to enlarge the margin between the "good" and "bad" sentences in a task-specific sense. It is trained end-to-end and can be widely applied to tasks that involve re-scoring of generated text. Compared with minimum-PPL training, our method gains up to 1.1 WER reduction for speech recognition and 1.0 BLEU increase for machine translation.