mlsanityData to 7 Oct 2026

Kad: A Framework for Proxy-based Test-time Alignment with Knapsack Approximation Deferral

arXiv 2510.27017 ↗First posted 30 Oct 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Several previous works concluded that the largest part of generation capabilities of large language models (LLM) are learned (early) during pre-training. However, LLMs still require further alignment to adhere to downstream task requirements and stylistic preferences, among other desired properties. As LLMs continue to scale in terms of size, the computational cost of alignment procedures increase prohibitively. In this work, we propose a novel approach to circumvent these costs via proxy-based test-time alignment, i.e. using guidance from a small aligned model. Our approach can be described as a token-specific cascading method, where the token-specific deferral rule is reduced to 0-1 knapsack problem. In this setting, we derive primal and dual approximations of the optimal deferral decision. We experimentally show the benefits of our method both in task performance and speculative decoding speed.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 45 references that are in the map. First posted on arXiv 30 Oct 2025.

Builds on

27 of 45 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2021Convex optimizationcited by 2,479
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2023LIMA: Less Is More for Alignmentcited by 1,080
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
15 more
  1. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  2. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  3. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  4. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 115
  5. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  6. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  7. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  8. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  9. 2017IDK Cascades: Fast Deep Learning by Learning not to Overthinkcited by 47
  10. 2020CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascadecited by 47
  11. 2022Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systemscited by 43
  12. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  13. 2024Nudging: Inference-time Alignment of LLMs via Guided Decodingcited by 22
  14. 2025Sample, Don't Search: Rethinking Test-Time Alignment for Language Modelscited by 21
  15. 2024On the Low-Rank Parametrization of Reward Models for Controlled Language Generationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026On the Rejection Criterion for Proxy-based Test-time Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024DeAL: Decoding-time Alignment for Large Language Models
  2. 2025Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner
  3. 2024Aligning Large Language Models with Representation Editing: A Control Perspective
  4. 2024Fast Best-of-N Decoding via Speculative Rejection
  5. 2026Asymptotic Universal Alignment: A New Alignment Framework via Test-Time Scaling
  6. 2025DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models
  7. 2024CodecLM: Aligning Language Models with Tailored Synthetic Data
  8. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Time