HY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editing
Not on the map. Why?
Abstract (arXiv)
Foundation models are transitioning from offline predictors to deployed systems expected to operate over long time horizons. In real deployments, objectives are not fixed: domains drift, user preferences evolve, and new tasks appear after the model has shipped. This elevates continual learning and instant personalization from optional features to core architectural requirements. Yet most adaptation pipelines still follow a static weight paradigm: after training (or after any adaptation step), inference executes a single parameter vector regardless of user intent, domain, or instance-specific constraints. This treats the trained or adapted model as a single point in parameter space. In heterogeneous and continually evolving regimes, distinct objectives can induce separated feasible regions over parameters, forcing any single shared update into compromise, interference, or overspecialization. As a result, continual learning and personalization are often implemented as repeated overwriting of shared weights, risking degradation of previously learned behaviors. We propose HY-WU (Weight Unleashing), a memory-first adaptation framework that shifts adaptation pressure away from overwriting a single shared parameter point. HY-WU implements functional (operator-level) memory as a neural module: a generator that synthesizes weight updates on-the-fly from the instance condition, yielding instance-specific operators without test-time optimization.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 75 of its 87 references that are in the map. First posted on arXiv 7 Mar 2026.
Builds on
75 of 87 references.
References · 75
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
38 more
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2021ViViT: A Video Vision Transformercited by 1,610
- 2019Generating Long Sequences with Sparse Transformerscited by 1,523
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
- 2016Progressive Neural Networkscited by 1,332
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
- 2017Continual Learning with Deep Generative Replaycited by 1,152
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2018Experience Replay for Continual Learningcited by 1,074
- 2016HyperNetworkscited by 1,003
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
- 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
- 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
- 2018Conditional Neural Processescited by 421
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
- 2019Dynamic Convolution: Attention over Convolution Kernelscited by 355
- 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
- 2019CondConv: Conditionally Parameterized Convolutions for Efficient Inferencecited by 273
- 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
50 most cited of 75 All 75 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.