mlsanityData to 7 Oct 2026

Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages

arXiv 2310.04799 ↗First posted 7 Oct 2023cs.CLcited by 52

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Recently, the development of open-source large language models (LLMs) has advanced rapidly. Nevertheless, due to data constraints, the capabilities of most open-source LLMs are primarily focused on English. To address this issue, we introduce the concept of textitchat vector to equip pre-trained language models with instruction following and human value alignment via simple model arithmetic. The chat vector is derived by subtracting the weights of a pre-trained base model (e.g. LLaMA2) from those of its corresponding chat model (e.g. LLaMA2-chat). By simply adding the chat vector to a continual pre-trained model's weights, we can endow the model with chat capabilities in new languages without the need for further training. Our empirical studies demonstrate the superior efficacy of the chat vector from three different aspects: instruction following, toxicity mitigation, and multi-turn dialogue. Moreover, to showcase the adaptability of our approach, we extend our experiments to encompass various languages, base models, and chat vectors. The results underscore the chat vector's simplicity, effectiveness, and wide applicability, making it a compelling solution for efficiently enabling conversational capabilities in pre-trained language models. Our code is available at https://github.com/aqweteddy/ChatVector.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 22 of its 39 references that are in the map. First posted on arXiv 7 Oct 2023.

Builds on

22 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  2. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  9. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  10. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  11. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  12. 2021Robust fine-tuning of zero-shot modelscited by 785
8 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  2. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  3. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  4. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  5. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  6. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  7. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 155
  8. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 21; 2026: 21.

52 papers.

Sort:

By area:

  1. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  2. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  3. 2024What Matters for Model Merging at Scale?cited by 52
  4. 2024Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editingcited by 23
  5. 2025Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuningcited by 16
  6. 2025Personality Vector: Modulating Personality of Large Language Models by Model Mergingcited by 16
  7. 2024Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognitioncited by 14
  8. 2024Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approachcited by 12
  9. 2024Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Mergingcited by 11
  10. 2024Fingerprint Vector: Enabling Scalable and Efficient Model Fingerprint Transfer via Vector Additioncited by 9
  11. 2025Shadow-FT: Tuning Instruct Model via Training on Paired Base Modelcited by 7
  12. 2024Adapting Chat Language Models Using Only Target Unlabeled Language Datacited by 6
38 more
  1. 2025Efficient Model Development through Fine-tuning Transfercited by 6
  2. 2025Adaptive Helpfulness-Harmlessness Alignment with Preference Vectorscited by 6
  3. 2025LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agentscited by 6
  4. 2024DogeRM: Equipping Reward Models with Domain Knowledge through Model Mergingcited by 5
  5. 2025Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledgecited by 5
  6. 2025Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Modelscited by 5
  7. 2025Investigating Task Arithmetic for Zero-Shot Information Retrievalcited by 4
  8. 2025Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updatescited by 4
  9. 2025Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacycited by 3
  10. 2025In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Modelscited by 3
  11. 2025V-Agent: An Interactive Video Search System Using Vision-Language Modelscited by 3
  12. 2025A Survey on Training-free Alignment of Large Language Modelscited by 2
  13. 2025Merging Continual Pretraining Models for Domain-Specialized LLMs: A Case Study in Financecited by 2
  14. 2025Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesiscited by 2
  15. 2026Bus-Conditioned Zero-Shot Trajectory Generation via Task Arithmeticcited by 2
  16. 2025PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolationcited by 1
  17. 2025Detoxification of Large Language Models through Output-layer Fusion with a Calibration Modelcited by 1
  18. 2026Patches of Nonlinearity: Instruction Vectors in Large Language Modelscited by 1
  19. 2026EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Trainingcited by 1
  20. 2026Multilinguality at the Edge: Developing Language Models for the Global Southcited by 1
  21. 2026DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Modelscited by 1
  22. 2025DisTaC: Conditioning Task Vectors via Distillation for Robust Model Mergingcited by 0
  23. 2025Guard Vector: Beyond English LLM Guardrails with Task-Vector Composition and Streaming-Aware Prefix SFTcited by 0
  24. 2025Democratizing LLM Efficiency: From Hyperscale Optimizations to Universal Deployabilitycited by 0
  25. 2026ELO: Efficient Layer-Specific Optimization for Continual Pretraining of Multilingual LLMscited by 0
  26. 2026BYOL: Bring Your Own Language Into LLMscited by 0
  27. 2026Controlling Chat Style in Language Models via Single-Direction Editingcited by 0
  28. 2026Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weightscited by 0
  29. 2026OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Trainingcited by 0
  30. 2026The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignmentcited by 0
  31. 2026DeCoVec: Building Decoding Space based Task Vector for Large Language Models via In-Context Learningcited by 0
  32. 2026Synthesizing Instruction-Tuning Datasets with Contrastive Decodingcited by 0
  33. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0
  34. 2026GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Modelscited by 0
  35. 2026A Gravitational Interpretation of Safety Reversion under Fine-Tuningcited by 0
  36. 2026Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shiftscited by 0
  37. 2026Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuningcited by 0
  38. 2026UpgradeBench: A Decision-Centric Benchmark for Upgrading Fine-Tuned LLM Specialistscited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Got Compute, but No Data: Lessons From Post-training a Finnish LLM
  2. 2024Why Not Transform Chat Large Language Models to Non-English?
  3. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
  4. 2024ChatGPT Alternative Solutions: Large Language Models Survey
  5. 2024Exploring Advanced Large Language Models with LLMsuite
  6. 2026HAL: Inducing Human-likeness in LLMs with Alignment
  7. 2024LAB: Large-Scale Alignment for ChatBots
  8. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignment