Platonic Task Arithmetic
Not on the map. Why?
Abstract (arXiv)
Distinct pre-trained models specialized for the same task converge to closely similar behavior, yet the parameter updates that produce it share no common coordinate system. Weight-space task arithmetic is therefore confined to a single model, and transporting an update between models requires a structural correspondence. Drawing on Plato's allegory of the cave, we hypothesize that these model-specific updates are shadows cast by one shared, model-agnostic object, the platonic task vector. To make it operational across models of different architectures, we introduce Universal Task Descriptors, matrices whose shape is independent of architecture and embedding dimension, which record a task's functional effect and admit addition and negation as ordinary matrix operations, and we transfer a descriptor into a target in two ways. A single least-squares solve returns a linear operator folded into the target's last layer, and a bank of such operators, one per source and task, realizes any composition as a signed sum of its entries. Alternatively, a low-rank adapter of the target's encoder is trained on the same objective at the price of one optimization per edit. Despite a model-specific residual comparable in norm to the shared component, transfer from another model retains 74 to 80 percent of the gain the target's own descriptors attain. Experiments across six model families, eight tasks and audio-text models confirm both realizations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 52 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
18 of 52 references.
References · 18
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2022Editing Models with Task Arithmeticcited by 1,120
- 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
- 2019Contrastive Representation Distillationcited by 748
- 2024The Platonic Representation Hypothesiscited by 382
- 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
- 2023Demystifying CLIP Datacited by 270
- 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
- 2022Relative representations enable zero-shot latent space communicationcited by 191
- 2023Task Arithmetic with LoRA for Continual Learningcited by 32
- 2025Understanding the Emergence of Multimodal Representation Alignmentcited by 31
6 more
- 2024On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigmcited by 26
- 2025LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptationcited by 11
- 2025Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMscited by 10
- 2025Proof of a perfect platonic representation hypothesiscited by 8
- 2026Transport and Merge: Cross-Architecture Merging for Large Language Modelscited by 7
- 2026Transporting Task Vectors across Different Architectures without Trainingcited by 6
Built on by
Not cited yet.