# PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness

arXiv: https://arxiv.org/abs/2410.07035 · HTML: https://mlsanity.com/paper/2410.07035/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [WikiHow: A Large Scale Text Summarization Dataset](https://mlsanity.com/paper/1810.09305/) (2018)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [Controlled Text Generation with Natural Language Instructions](https://mlsanity.com/paper/2304.14293/) (2023)
- [Evaluating Large Language Models on Controlled Generation Tasks](https://mlsanity.com/paper/2310.14542/) (2023)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [Contextual Position Encoding: Learning to Count What's Important](https://mlsanity.com/paper/2405.18719/) (2024)
- [Weaver: Foundation Models for Creative Writing](https://mlsanity.com/paper/2401.17268/) (2024)

## Built on by (most cited first, 9 of 9)

- [From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MARKERGEN](https://mlsanity.com/paper/2502.13544/) (2025)
- [How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting](https://mlsanity.com/paper/2507.22923/) (2025)
- [Length Representations in Large Language Models](https://mlsanity.com/paper/2507.20398/) (2025)
- [BoundRL: Efficient Structured Text Segmentation through Reinforced Boundary Generation](https://mlsanity.com/paper/2510.20151/) (2025)
- [LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models](https://mlsanity.com/paper/2603.19255/) (2026)
- [TiCo: Time-Controllable Spoken Dialogue Model](https://mlsanity.com/paper/2603.22267/) (2026)
- [Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization](https://mlsanity.com/paper/2509.09852/) (2025)
- [Towards Faithful Sentimental Image Captioning via Evidence-Aware Multi-Agent Reasoning](https://mlsanity.com/paper/2607.25789/) (2026)
- [Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models](https://mlsanity.com/paper/2609.32706/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) demonstrate impressive capabilities across various domains, including role-playing, creative writing, mathematical reasoning, and coding. Despite these advancements, LLMs still encounter challenges with length control, frequently failing to adhere to specific length constraints due to their token-level operations and insufficient training on data with strict length limitations. We identify this issue as stemming from a lack of positional awareness and propose novel approaches--PositionID Prompting and PositionID Fine-Tuning--to address it. These methods enhance the model's ability to continuously monitor and manage text length during generation. Additionally, we introduce PositionID CP Prompting to enable LLMs to perform copy and paste operations accurately. Furthermore, we develop two benchmarks for evaluating length control and copy-paste abilities. Our experiments demonstrate that our methods significantly improve the model's adherence to length constraints and copy-paste accuracy without compromising response quality.