# Consensus-Aligned Neuron Efficient Fine-Tuning Large Language Models for Multi-Domain Machine Translation

arXiv: https://arxiv.org/abs/2602.05694 · HTML: https://mlsanity.com/paper/2602.05694/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [LLaMA Pro: Progressive LLaMA with Block Expansion](https://mlsanity.com/paper/2401.02415/) (2024)
- [Neurons in Large Language Models: Dead, N-gram, Positional](https://mlsanity.com/paper/2309.04827/) (2023)
- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons](https://mlsanity.com/paper/2308.13198/) (2023)
- [What does the Knowledge Neuron Thesis Have to do with Knowledge?](https://mlsanity.com/paper/2405.02421/) (2024)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Fine-tuning Large Language Models for Domain-specific Machine Translation](https://mlsanity.com/paper/2402.15061/) (2024)
- [Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons](https://mlsanity.com/paper/2407.06488/) (2024)
- [Importance-based Neuron Allocation for Multilingual Neural Machine Translation](https://mlsanity.com/paper/2107.06569/) (2021)
- [An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs](https://mlsanity.com/paper/2406.12288/) (2024)
- [Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translation](https://mlsanity.com/paper/2404.11201/) (2024)
- [MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation](https://mlsanity.com/paper/1912.05467/) (2019)
- [Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning](https://mlsanity.com/paper/2410.02631/) (2024)
- [Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning](https://mlsanity.com/paper/2412.05586/) (2024)
- [Leveraging Domain Knowledge at Inference Time for LLM Translation: Retrieval versus Generation](https://mlsanity.com/paper/2503.05010/) (2025)
- [Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble](https://mlsanity.com/paper/2412.20637/) (2024)
- [Multilingual Knowledge Graph Completion via Efficient Multilingual Knowledge Sharing](https://mlsanity.com/paper/2510.07736/) (2025)

## Abstract (arXiv)

Multi-domain machine translation (MDMT) aims to build a unified model capable of translating content across diverse domains. Despite the impressive machine translation capabilities demonstrated by large language models (LLMs), domain adaptation still remains a challenge for LLMs. Existing MDMT methods such as in-context learning and parameter-efficient fine-tuning often suffer from domain shift, parameter interference and limited generalization. In this work, we propose a neuron-efficient fine-tuning framework for MDMT that identifies and updates consensus-aligned neurons within LLMs. These neurons are selected by maximizing the mutual information between neuron behavior and domain features, enabling LLMs to capture both generalizable translation patterns and domain-specific nuances. Our method then fine-tunes LLMs guided by these neurons, effectively mitigating parameter interference and domain-specific overfitting. Comprehensive experiments on three LLMs across ten German-English and Chinese-English translation domains evidence that our method consistently outperforms strong PEFT baselines on both seen and unseen domains, achieving state-of-the-art performance.