mlsanityData to 7 Oct 2026

Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM

arXiv 2605.01973 ↗First posted 3 May 2026cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Conventional LLMs may suffer from corpus heterogeneity and subtle condition changes. While finetuning can create the catastrophe forgetting issue, application of meta-learning on LLMs is also limited due to its complexity and scalability. In this paper, we activate the meta-signal of β within the SwiGLU blocks, resulting in a meta-gating mechanism that adaptively adjusts the nonlinearity of FFN. A hypernetwork is employed which dynamically produces β on textual conditions, providing meta-controllability on LLMs. By testing on different condition types such as task, domain, persona, and style, our method outperforms finetuning and meta-learning baselines, and can generalize reasonably on unseen tasks, condition types, or instructions. Our code can be found in https://github.com/AaronJi/MeGan.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 64 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

49 of 64 references.

References · 49

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2023Mistral 7Bcited by 3,065
  10. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  12. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
37 more
  1. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  3. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  4. 2016HyperNetworkscited by 1,003
  5. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  8. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  9. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  10. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  11. 2021MetaICL: Learning to Learn In Contextcited by 450
  12. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  13. 2021Towards Emotional Support Dialog Systemscited by 294
  14. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  15. 2023Language Modeling Is Compressioncited by 225
  16. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  17. 2021Meta-learning via Language Model In-context Tuningcited by 163
  18. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  19. 2022The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systemscited by 92
  20. 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
  21. 2024Do LLMs write like humans? Variation in grammatical and rhetorical stylescited by 62
  22. 2023Meta-in-context learning in large language modelscited by 59
  23. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  24. 2021AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarizationcited by 51
  25. 2025Text-to-LoRA: Instant Transformer Adaptioncited by 50
  26. 2024Compress then Serve: Serving Thousands of LoRA Adapters with Little Overheadcited by 24
  27. 2026SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Passcited by 22
  28. 2019Learning to Customize Model Structures for Few-shot Dialogue Generation Taskscited by 15
  29. 2022Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptationcited by 14
  30. 2024Meta In-Context Learning Makes Large Language Models Better Zero and Few-Shot Relation Extractorscited by 13
  31. 2025Comparing LLM-generated and human-authored news text using formal syntactic theorycited by 11
  32. 2024MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learningcited by 10
  33. 2025Substance over Style: Evaluating Proactive Conversational Coaching Agentscited by 6
  34. 2025Dissecting Persona-Driven Reasoning in Language Models via Activation Patchingcited by 5
  35. 2020MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learningcited by 4
  36. 2023Meta-Gating Framework for Fast and Continuous Resource Optimization in Dynamic Wireless Environmentscited by 4
  37. 2025In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Modelscited by 3

Built on by

Not cited yet.