mlsanityData to 14 Oct 2026

Shuttle Between the Instructions and the Parameters of Large Language Models

arXiv 2502.02315 ↗First posted 4 Feb 2025cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The interaction with Large Language Models (LLMs) through instructions has been extensively investigated in the research community. While instructions have been widely used as the guidelines for task solving, this paper further notices that both instructions and parameters are the compression of task data. Therefore, they could be strongly correlated and can be learned to predict one from the other. This paper proposes a novel neural network framework, SHIP (Shuttle between the Instructions and the Parameters), to model and learn the mutual mappings between the instructions and the parameters of LLMs. We verify that SHIP can effectively map one of the instructions/parameters to the other by evaluating it on the tasks of instruction deduction and induction. The results show that SHIP performs better than existing baseline methods in terms of deductive capabilities while significantly surpassing them in inductive capabilities. Moreover, SHIP can effectively combine the two mapping processes to perform excellent inductive reasoning. The code and data for this paper are released at https://anonymous.4open.science/r/Shuttle-Between-Instructions-Parameters/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 26 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

22 of 26 references.

References · 22

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  6. 2016Deep Variational Information Bottleneckcited by 930
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  8. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  9. 2015Generating Sentences from a Continuous Spacecited by 612
  10. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  11. 2023Language Modeling Is Compressioncited by 225
  12. 2023Large Language Models as General Pattern Machinescited by 197
10 more
  1. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  2. 2024InFoBench: Evaluating Instruction Following Ability in Large Language Modelscited by 135
  3. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
  4. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  5. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasonerscited by 63
  6. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 55
  7. 2022HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisationcited by 28
  8. 2024Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Modelscited by 23
  9. 2024ItD: Large Language Models Can Teach Themselves Induction through Deductioncited by 8
  10. 2024From Instance Training to Instruction Learning: Task Adapters Generation from Instructionscited by 4

Built on by

Not cited yet.