mlsanityData to 7 Oct 2026

Positional Encoding to Control Output Sequence Length

arXiv 1904.07418 ↗First posted 16 Apr 2019cs.CLcited by 56

Not on the map. Why?

Abstract (arXiv)

Neural encoder-decoder models have been successful in natural language generation tasks. However, real applications of abstractive summarization must consider additional constraint that a generated summary should not exceed a desired length. In this paper, we propose a simple but effective extension of a sinusoidal positional encoding (Vaswani et al., 2017) to enable neural encoder-decoder model to preserves the length constraint. Unlike in previous studies where that learn embeddings representing each length, the proposed method can generate a text of any length even if the target length is not present in training data. The experimental results show that the proposed method can not only control the generation length but also improve the ROUGE scores.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 10; 2023: 7; 2024: 10; 2025: 10; 2026: 4.

56 papers.

Sort:

By area:

  1. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Costcited by 108
  2. 2021Position Information in Transformers: An Overviewcited by 103
  3. 2022Transformers in Time-series Analysis: A Tutorialcited by 45
  4. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agentscited by 39
  5. 2021SHAPE: Shifted Absolute Position Embedding for Transformerscited by 37
  6. 2021Controllable Abstractive Dialogue Summarization with Sketch Supervisioncited by 27
  7. 2024Benchmarking Large Language Models on Controllable Generation under Diversified Instructionscited by 26
  8. 2022Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimationcited by 22
  9. 2022VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbingcited by 20
  10. 2024Prompt-Based Length Controlled Generation with Multiple Control Typescited by 19
  11. 2021Machine Translation Verbosity Control for Automatic Dubbingcited by 18
  12. 2021Isometric MT: Neural Machine Translation for Automatic Dubbingcited by 18
38 more
  1. 2023Controllable Text Summarization: Unraveling Challenges, Approaches, and Prospects -- A Surveycited by 18
  2. 2021Rethinking Perturbations in Encoder-Decoders for Fast Trainingcited by 17
  3. 2022B2T Connection: Serving Stability and Performance in Deep Transformerscited by 16
  4. 2021LenAtten: An Effective Length Controlling Unit For Text Summarizationcited by 14
  5. 2021Isochrony-Aware Neural Machine Translation for Automatic Dubbingcited by 14
  6. 2023Prompt-Based Length Controlled Generation with Reinforcement Learningcited by 14
  7. 2022A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarizationcited by 13
  8. 2021Lost in Interpreting: Speech Translation from Source or Interpreter?cited by 10
  9. 2022Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale Graphcited by 10
  10. 2024Length Controlled Generation for Black-box LLMscited by 8
  11. 2022Factorizing Content and Budget Decisions in Abstractive Summarization of Long Documentscited by 7
  12. 2021Neural Class Expression Synthesiscited by 6
  13. 2021Unifying Cross-lingual Summarization and Machine Translation with Compression Ratecited by 5
  14. 2024Knowledge of Pretrained Language Models on Surface Information of Tokenscited by 5
  15. 2023Summarization with Precise Length Controlcited by 4
  16. 2024InstructCMP: Length Control in Sentence Compression through Instruction-based Large Language Modelscited by 4
  17. 2026Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulationcited by 4
  18. 2021Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translationcited by 3
  19. 2022AppTek's Submission to the IWSLT 2022 Isometric Spoken Language Translation Taskcited by 3
  20. 2025Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimizationcited by 3
  21. 2022Text-Driven Video Acceleration: A Weakly-Supervised Reinforcement Learning Methodcited by 2
  22. 2022Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attentioncited by 2
  23. 2023How Transferable are Attribute Controllers on Pretrained Multilingual Translation Models?cited by 2
  24. 2024Fine-grained length controllable video captioning with ordinal embeddingscited by 2
  25. 2024Hansel: Output Length Controlling Framework for Large Language Modelscited by 2
  26. 2025Ecomap: Sustainability-Driven Optimization of Multi-Tenant DNN Execution on Edge Serverscited by 2
  27. 2025Considering Length Diversity in Retrieval-Augmented Summarizationcited by 2
  28. 2025Prompting LLMs: Length Control for Isometric Machine Translationcited by 2
  29. 2021An Empirical Study of Extrapolation in Text Generation with Scalar Controlcited by 1
  30. 2021Goal-driven text descriptions for imagescited by 1
  31. 2021Cross-Register Projection for Headline Part of Speech Taggingcited by 1
  32. 2023The Locality and Symmetry of Positional Encodingscited by 1
  33. 2023Unsupervised Extractive Summarization with Learnable Length Control Strategiescited by 1
  34. 2025UltraGen: Extremely Fine-grained Controllable Generation via Attribute Reconstruction and Global Preference Optimizationcited by 1
  35. 2025Controlling Summarization Length Through EOS Token Weightingcited by 1
  36. 2025Length Representations in Large Language Modelscited by 1
  37. 2025On the Interplay between Positional Encodings, Morphological Complexity, and Word Order Flexibilitycited by 1
  38. 2023Weakly Supervised Headline Dependency Parsingcited by 0

50 most cited of 56 All 56 (JSON)