mlsanityData to 7 Oct 2026

Position Engineering: Boosting Large Language Models through Positional Information Manipulation

arXiv 2404.11216 ↗First posted 17 Apr 2024cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

The performance of large language models (LLMs) is significantly influenced by the quality of the prompts provided. In response, researchers have developed enormous prompt engineering strategies aimed at modifying the prompt text to enhance task performance. In this paper, we introduce a novel technique termed position engineering, which offers a more efficient way to guide large language models. Unlike prompt engineering, which requires substantial effort to modify the text provided to LLMs, position engineering merely involves altering the positional information in the prompt without modifying the text itself. We have evaluated position engineering in two widely-used LLM scenarios: retrieval-augmented generation (RAG) and in-context learning (ICL). Our findings show that position engineering substantially improves upon the baseline in both cases. Position engineering thus represents a promising new strategy for exploiting the capabilities of large language models.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

24 of 34 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2023Mistral 7Bcited by 3,065
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
12 more
  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  2. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  6. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  7. 2022A Survey on In-context Learningcited by 717
  8. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  9. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  10. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  11. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  12. 2023Survival of the Most Influential Prompts: Efficient Black-Box Prompt Search via Clustering and Pruningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 4; 2026: 3.

9 papers.

  1. 2024Attention Instruction: Amplifying Attention in the Middle via Promptingcited by 6
  2. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  3. 2025Less LLM, More Documents: Searching for Improved RAGcited by 4
  4. 2025BEE-RAG: Balanced Entropy Engineering for Retrieval-Augmented Generationcited by 3
  5. 2024Mitigate Position Bias in Large Language Models via Scaling a Single Dimensioncited by 2
  6. 2026Self-Describing Structured Data with Dual-Layer Guidance: A Lightweight Alternative to RAG for Precision Retrieval in Large-Scale LLM Knowledge Navigationcited by 1
  7. 2025Path Pooling: Training-Free Structure Enhancement for Efficient Knowledge Graph Retrieval-Augmented Generationcited by 0
  8. 2025CoRAG: Collaborative Retrieval-Augmented Generationcited by 0
  9. 2026More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffoldingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Eliminating Position Bias of Language Models: A Mechanistic Approach
  2. 2025A Survey of Context Engineering for Large Language Models
  3. 2026Do Large Language Models Plan Answer Positions? Position Bias in Multiple-Choice Question Generation
  4. 2026The Position Curse: LLMs Struggle to Locate the Last Few Items in a List
  5. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
  6. 2023Retrieval meets Long Context Large Language Models
  7. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  8. 2024Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models