mlsanityData to 7 Oct 2026

Tuning Language Models by Proxy

arXiv 2401.08565 ↗First posted 16 Jan 2024cs.CLcited by 88

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite the general capabilities of large pretrained language models, they consistently benefit from further adaptation to better achieve desired behaviors. However, tuning these models has become increasingly resource-intensive, or impossible when model weights are private. We introduce proxy-tuning, a lightweight decoding-time algorithm that operates on top of black-box LMs to achieve the same end as direct tuning, but by accessing only its predictions over the output vocabulary, not its parameters. Our method tunes a smaller LM, then applies the difference between the predictions of the small tuned and untuned LMs to shift the original predictions of the larger untuned model in the direction of tuning, while retaining the benefits of larger-scale pretraining. In experiments, when we apply proxy-tuning to Llama2-70B using proxies of only 7B size, we can close 88% of the gap between Llama2-70B and its truly-tuned chat version, when evaluated across knowledge, reasoning, and safety benchmarks. We then demonstrate the generality of proxy-tuning by applying it to domain adaptation on code, and task-specific finetuning on question-answering and math problems. Finally, we show how to proxy-tune a truly black-box LM, GPT-3.5, for temporal adaptation, increasing its knowledge about recent events. Our work demonstrates the promise of using small tuned LMs to efficiently customize large, potentially proprietary LMs through decoding-time guidance.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Builds on 47 of its 48 references that are in the map. First posted on arXiv 16 Jan 2024.

Builds on

47 of 48 references.

References · 47

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
35 more
  1. 2020Learning to summarize from human feedbackcited by 2,562
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  3. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  4. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  5. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  6. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  7. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  8. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
  9. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  10. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  11. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  12. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  13. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  14. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  15. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  16. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  17. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  18. 2023The False Promise of Imitating Proprietary LLMscited by 254
  19. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  20. 2022RealTime QA: What's the Answer Right Now?cited by 239
  21. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  22. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  23. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  24. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  25. 2023An Emulator for Fine-Tuning Large Language Models using Small Language Modelscited by 61
  26. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  27. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  28. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  29. 2023PREADD: Prefix-Adaptive Decoding for Controlled Text Generationcited by 29
  30. 2023The Benefits of Bad Advice: Autocontrastive Decoding across Model Layerscited by 28
  31. 2023CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Modelscited by 26
  32. 2023In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuningcited by 19
  33. 2019Domain Differential Adaptation for Neural Machine Translationcited by 10
  34. 2023David helps Goliath: Inference-Time Collaboration Between Small Specialized and Large General Diffusion LMscited by 6
  35. 2022Twist Decoding: Diverse Generators Guide Each Othercited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 33; 2025: 25; 2026: 30.

88 papers.

Sort:

By area:

  1. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  2. 2024Decoding-Time Language Model Alignment with Multiple Objectivescited by 90
  3. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Differencecited by 85
  4. 2024Decoding-time Realignment of Language Modelscited by 67
  5. 2024Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimizationcited by 56
  6. 2024Learning or Self-aligning? Rethinking Instruction Fine-tuningcited by 43
  7. 2024Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B Peoplecited by 38
  8. 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 35
  9. 2024Offset Unlearning for Large Language Modelscited by 33
  10. 2024From Distributional to Overton Pluralism: Investigating Large Language Model Alignmentcited by 33
  11. 2024Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Modelscited by 32
  12. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
38 more
  1. 2024Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligencecited by 27
  2. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Modelscited by 25
  3. 2024Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Modelcited by 22
  4. 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 22
  5. 2024Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrastcited by 19
  6. 2024Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignmentcited by 19
  7. 2024Inference-Time Language Model Alignment via Integrated Value Guidancecited by 19
  8. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  9. 2024Learning From Correctness Without Prompting Makes LLM Efficient Reasonercited by 15
  10. 2024MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usabilitycited by 14
  11. 2025Qsharp: Provably Optimal Distributional RL for LLM Post-Trainingcited by 14
  12. 2024Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decodingcited by 12
  13. 2025CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steeringcited by 12
  14. 2024On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusioncited by 9
  15. 2024Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learningcited by 9
  16. 2025Don't Throw Away Your Pretrained Modelcited by 9
  17. 2025LaSeR: Reinforcement Learning with Last-Token Self-Rewardingcited by 9
  18. 2024ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantationcited by 8
  19. 2024Towards Building Specialized Generalist AI with System 1 and System 2 Fusioncited by 8
  20. 2025Robust Multi-Objective Controlled Decoding of Large Language Modelscited by 8
  21. 2025Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLMcited by 8
  22. 2025ProxyThinker: Test-Time Guidance through Small Visual Reasonerscited by 8
  23. 2024The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insightscited by 7
  24. 2024PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patchescited by 7
  25. 2025Shadow-FT: Tuning Instruct Model via Training on Paired Base Modelcited by 7
  26. 2025Sampling from Your Language Model One Byte at a Timecited by 7
  27. 2026RelayLLM: Efficient Reasoning via Collaborative Decodingcited by 7
  28. 2026Delta Attention Residualscited by 7
  29. 2026Weak-to-Strong On-Policy Distillationcited by 7
  30. 2024Learning to Correct for QA Reasoning with Black-box LLMscited by 6
  31. 2024GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Modelscited by 6
  32. 2024Humanizing the Machine: Proxy Attacks to Mislead LLM Detectorscited by 6
  33. 2026SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generationcited by 6
  34. 2025Sysformer: Safeguarding Frozen Large Language Models with Adaptive System Promptscited by 5
  35. 2026MoCo: A One-Stop Shop for Model Collaboration Researchcited by 5
  36. 2025Logits are All We Need to Adapt Closed Modelscited by 4
  37. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  38. 2026ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decodingcited by 4

50 most cited of 88 All 88 (JSON)