# Text2Weight: Bridging Natural Language and Neural Network Weight Spaces

arXiv: https://arxiv.org/abs/2508.13633 · HTML: https://mlsanity.com/paper/2508.13633/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [Equivariant Architectures for Learning in Deep Weight Spaces](https://mlsanity.com/paper/2301.12780/) (2023)
- [Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights](https://mlsanity.com/paper/2209.14733/) (2022)
- [Graph Metanetworks for Processing Diverse Neural Architectures](https://mlsanity.com/paper/2312.04501/) (2023)
- [Diffusion-Based Neural Network Weights Generation](https://mlsanity.com/paper/2402.18153/) (2024)
- [DiffLoRA: Generating Personalized Low-Rank Adaptation Weights with Diffusion](https://mlsanity.com/paper/2408.06740/) (2024)
- [Text-to-Model: Text-Conditioned Neural Network Diffusion for Train-Once-for-All Personalization](https://mlsanity.com/paper/2405.14132/) (2024)
- [Efficient Training with Denoised Neural Weights](https://mlsanity.com/paper/2407.11966/) (2024)

## Built on by (most cited first, 5 of 5)

- [MemoGen: Can Past Experience Improve Future Text-to-Image Generation?](https://mlsanity.com/paper/2606.03243/) (2026)
- [AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise](https://mlsanity.com/paper/2606.01053/) (2026)
- [Before the Body Moves: Learning Anticipatory Joint Intent for Language-Conditioned Humanoid Control](https://mlsanity.com/paper/2605.14417/) (2026)
- [Traffic-CBM: A Structurally Interpretable Multimodal Framework for Encrypted Traffic Classification](https://mlsanity.com/paper/2606.29909/) (2026)
- [WeightCLIP: Aligning Datasets and Models for Weight Space Learning](https://mlsanity.com/paper/2607.03551/) (2026)

## Abstract (arXiv)

How far are we really from automatically generating neural networks? While neural network weight generation shows promise, current approaches struggle with generalization to unseen tasks and practical application exploration. To address this, we propose T2W, a diffusion transformer framework that generates task-specific weights conditioned on natural language descriptions. T2W hierarchically processes network parameters into uniform blocks, integrates text embeddings from CLIP via a prior attention mechanism, and employs adversarial training with weight-space augmentation to enhance generalization. Experiments on Cifar100, Caltech256, and TinyImageNet demonstrate T2W's ability to produce high-quality weights for unseen tasks, outperforming optimization-based initialization and enabling novel applications such as weight enhancement and text-guided model fusion. Our work bridges textual semantics with weight-space dynamics, supported by an open-source dataset of text-weight pairs, advancing the practicality of generative models in neural network parameter synthesis. Our code is available on Github.