mlsanityData to 14 Oct 2026

Sequence-to-Sequence Learning as Beam-Search Optimization

arXiv 1606.02960 ↗First posted 9 Jun 2016cs.CL cs.LG cs.NEcited by 127

Not on the map. Why?

Abstract (arXiv)

Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work, we introduce a model and beam-search training scheme, based on the work of Daume III and Marcu (2005), that extends seq2seq to learn global sequence scores. This structured approach avoids classical biases associated with local training and unifies the training loss with the test-time usage, while preserving the proven model architecture of seq2seq and its efficient training approach. We show that our system outperforms a highly-optimized attention-based seq2seq system and other baselines on three different sequence to sequence tasks: word ordering, parsing, and machine translation.

Key facts. Cited by 127 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 3; 2021: 37; 2022: 27; 2023: 25; 2024: 13; 2025: 10; 2026: 11.

127 papers.

Sort:

By area:

  1. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  2. 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
  3. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  4. 2022End-to-end symbolic regression with transformerscited by 203
  5. 2024The Road Less Scheduledcited by 163
  6. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  7. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 132
  8. 2023Learning-Rate-Free Learning by D-Adaptationcited by 113
  9. 2021Deep Policy Dynamic Programming for Vehicle Routing Problemscited by 94
  10. 2021Survey of Low-Resource Machine Translationcited by 92
  11. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  12. 2023Transformer-based Planning for Symbolic Regressioncited by 79
38 more
  1. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 74
  2. 2021NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentationcited by 73
  3. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  4. 2020Text Generation by Learning from Demonstrationscited by 64
  5. 2022Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformercited by 59
  6. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  7. 2023Learning to Generate Better Than Your LLMcited by 49
  8. 2023Optimal Linear Decay Learning Rate Schedules and Further Refinementscited by 43
  9. 2021Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastingscited by 37
  10. 2023Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignmentcited by 35
  11. 2022SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parsercited by 32
  12. 2025Parallel Scaling Law for Language Modelscited by 30
  13. 2021Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimizationcited by 28
  14. 2023Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspectivecited by 26
  15. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  16. 2022Medical Image Captioning via Generative Pretrained Transformerscited by 20
  17. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  18. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  19. 2023Aligning Neural Machine Translation Models: Human Feedback in Training and Inferencecited by 19
  20. 2021BERTTune: Fine-Tuning Neural Machine Translation with BERTScorecited by 18
  21. 2021Modelling Latent Translations for Cross-Lingual Transfercited by 16
  22. 2021Outlining and Filling: Hierarchical Query Graph Generation for Answering Complex Questions over Knowledge Graphscited by 16
  23. 2022Video Question Answering with Iterative Video-Text Co-Tokenizationcited by 16
  24. 2023Formally Specifying the High-Level Behavior of LLM-Based Agentscited by 15
  25. 2023Better Language Models of Code through Self-Improvementcited by 14
  26. 2022BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsingcited by 13
  27. 2021Reward Optimization for Neural Machine Translation with Learned Metricscited by 12
  28. 2021Sequence to General Tree: Knowledge-Guided Geometry Word Problem Solvingcited by 11
  29. 2021Scheduled Sampling Based on Decoding Steps for Neural Machine Translationcited by 11
  30. 2022Teacher Forcing Recovers Reward Functions for Text Generationcited by 11
  31. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!cited by 11
  32. 2024Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappingscited by 11
  33. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMscited by 11
  34. 2021Towards User-Driven Neural Machine Translationcited by 10
  35. 2021Semi-Autoregressive Image Captioningcited by 10
  36. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  37. 2018Learning Beam Search Policies via Imitation Learningcited by 9
  38. 2022BashExplainer: Retrieval-Augmented Bash Code Comment Generation based on Fine-tuned CodeBERTcited by 9

50 most cited of 127 All 127 (JSON)