mlsanityData to 14 Oct 2026

DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models

arXiv 2607.26891 ↗First posted 29 Jul 2026cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Sequence labeling is a fine-grained information extraction task, yet existing large language model-based approaches suffer from insufficient domain alignment and low inference efficiency. To address these issues, we propose DIRECT, a framework that addresses these issues through training-time optimization and inference-time rectification. Specifically, DIRECT performs Direct Preference Optimization (DPO) after supervised fine-tuning to strengthen task alignment with human preferences, and introduces a controlled decoding process that enforces fixed output formats and restricts predictions to candidate sets. To further improve efficiency, a template-filling mechanism requires the model to generate only label tokens while reusing prefixed content through the KV Cache, thus reducing redundant computation. Experimental results on eight datasets demonstrate that DIRECT achieves significant improvements in both performance and efficiency compared to existing methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 23 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

16 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  5. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  6. 2022Unified Structure Generation for Universal Information Extractioncited by 221
  7. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 170
  8. 2023GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extractioncited by 119
  9. 2018Chinese NER Using Lattice LSTMcited by 81
  10. 2016Improving Coreference Resolution by Learning Entity-Level Distributed Representationscited by 50
  11. 2023Universal Information Extraction as Unified Semantic Matchingcited by 48
  12. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 23
3 more
  1. 2021Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategiescited by 21
  2. 2024Rethinking Negative Instances for Generative Named Entity Recognitioncited by 13
  3. 2024Graphusion: Leveraging Large Language Models for Scientific Knowledge Graph Fusion and Construction in NLP Educationcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Alignment-Aware Decoding
  2. 2024ICDPO: Effectively Borrowing Alignment Capability of Others via In-context Direct Preference Optimization
  3. 2025Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding
  4. 2024A Thorough Examination of Decoding Methods in the Era of LLMs
  5. 2024DeAL: Decoding-time Alignment for Large Language Models
  6. 2025Bringing Emerging Architectures to Sequence Labeling in NLP
  7. 2024Nudging: Inference-time Alignment of LLMs via Guided Decoding
  8. 2024Cascade Reward Sampling for Efficient Decoding-Time Alignment