mlsanityData to 7 Oct 2026

LOGO -- Long cOntext aliGnment via efficient preference Optimization

arXiv 2410.18533 ↗First posted 24 Oct 2024cs.CL cs.AIcited by 9

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Long-context models(LCMs) have shown great potential in processing long input sequences(even more than 100M tokens) conveniently and effectively. With significant progress, recent research has pointed out that LCMs can accurately locate token-level salient information within the context. Yet, the generation performance of these LCMs is far from satisfactory and might result in misaligned responses, such as hallucinations. To enhance the generation capability of LCMs, existing works have investigated the effects of data size and quality for both pre-training and instruction tuning. Though achieving meaningful improvement, previous methods fall short in either effectiveness or efficiency. In this paper, we introduce LOGO(Long cOntext aliGnment via efficient preference Optimization), a training strategy that first introduces preference optimization for long-context alignment. To overcome the GPU memory-bound issue caused by the long sequence, LOGO employs a reference-free preference optimization strategy and adopts a position synthesis method to construct the training data. By training with only 0.3B data on a single 8×A800 GPU machine for 16 hours, LOGO allows the Llama-3-8B-Instruct-80K model to achieve comparable performance with GPT-4 in real-world long-context tasks while preserving the model's original capabilities on other tasks, e.g., language modeling and MMLU. Moreover, LOGO can extend the model's context window size while enhancing its generation performance.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 43 of its 52 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

43 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Reward23 May
  2. 2024LongAlign: A Recipe for Long Context Alignment of Large Language Models31 Jan
  3. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation16 Jan
  4. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
27 more
  1. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  2. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  3. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  4. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  5. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  6. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  7. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  8. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
  9. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  10. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  11. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  12. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  13. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 243
  14. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 219
  15. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 183
  16. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  17. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 133
  18. 2023Randomized Positional Encodings Boost Length Generalization of Transformerscited by 114
  19. 2023PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Trainingcited by 109
  20. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
  21. 2024LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAcited by 78
  22. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  23. 2023On Context Utilization in Summarization with Large Language Modelscited by 30
  24. 2024Long Context Alignment with Short Instructions and Synthesized Positionscited by 22
  25. 2024Extending Llama-3's Context Ten-Fold Overnightcited by 15
  26. 2024SinkLoRA: Enhanced Efficiency and Chat Capabilities for Long-Context Large Language Modelscited by 8
  27. 2024Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Costcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 1.

9 papers.

  1. 2025SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimizationcited by 10
  2. 2025When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Frameworkcited by 8
  3. 2025LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructionscited by 6
  4. 2025Revisiting Long-context Modeling from Context Denoising Perspectivecited by 5
  5. 2025LongRM: Revealing and Unlocking the Context Boundary of Reward Modelingcited by 5
  6. 2025A Survey of LLM Alignment: Instruction Understanding, Intention Reasoning, and Reliable Generationcited by 3
  7. 2025Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimizationcited by 3
  8. 2025Flora: Effortless Context Construction to Arbitrary Length and Scalecited by 2
  9. 2026LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimizationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Long-Short Alignment for Effective Long-Context Modeling in LLMs
  2. 2026ECLIPTICA -- A Framework for Switchable LLM Alignment via CITA - Contrastive Instruction-Tuned Alignment
  3. 2025LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization
  4. 2025Aligning Multimodal LLM with Human Preference: A Survey
  5. 2024Understanding Alignment in Multimodal LLMs: A Comprehensive Study
  6. 2025A Comprehensive Evaluation framework of Alignment Techniques for LLMs
  7. 2024DeAL: Decoding-time Alignment for Large Language Models
  8. 2025LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization