# Unlocking Emergent Modularity in Large Language Models

arXiv: https://arxiv.org/abs/2310.10908 · HTML: https://mlsanity.com/paper/2310.10908/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 54 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts](https://mlsanity.com/paper/2111.02358/) (2021)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints](https://mlsanity.com/paper/2212.05055/) (2022)
- [Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts](https://mlsanity.com/paper/2206.02770/) (2022)
- [Transformer-Patcher: One Mistake worth One Neuron](https://mlsanity.com/paper/2301.09785/) (2023)
- [A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms](https://mlsanity.com/paper/1901.10912/) (2019)
- [MoEfication: Transformer Feed-forward Layers are Mixtures of Experts](https://mlsanity.com/paper/2110.01786/) (2021)
- [Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks](https://mlsanity.com/paper/2010.02066/) (2020)
- [Scaling Vision-Language Models with Sparse Mixture of Experts](https://mlsanity.com/paper/2303.07226/) (2023)

## Built on by (most cited first, 17 of 17)

- [Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models](https://mlsanity.com/paper/2501.11873/) (2025)
- [Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models](https://mlsanity.com/paper/2404.05567/) (2024)
- [Post-hoc Reward Calibration: A Case Study on Length Bias](https://mlsanity.com/paper/2409.17407/) (2024)
- [Layerwise Recurrent Router for Mixture-of-Experts](https://mlsanity.com/paper/2408.06793/) (2024)
- [Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference](https://mlsanity.com/paper/2312.10193/) (2023)
- [Position: Weight Space Should Be a First-Class Generative AI Modality](https://mlsanity.com/paper/2605.18632/) (2026)
- [Mixture of Hidden-Dimensions Transformer](https://mlsanity.com/paper/2412.05644/) (2024)
- [Steering Information Utility in Key-Value Memory for Language Model Post-Training](https://mlsanity.com/paper/2507.05158/) (2025)
- [Tiny Brains, Giant Impact: Uncovering the Keystone Neurons of LLM with Just a Few Prompts](https://mlsanity.com/paper/2605.24846/) (2026)
- [Neuronal Group Communication for Efficient Neural representation](https://mlsanity.com/paper/2510.16851/) (2025)
- [GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models](https://mlsanity.com/paper/2603.13418/) (2026)
- [Learning Emergent Modular Representations in Multi-modality Medical Vision Foundation Models](https://mlsanity.com/paper/2605.21861/) (2026)
- [STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning](https://mlsanity.com/paper/2606.08814/) (2026)
- [PADD: Path-Aligned Decompression Distillation for Non-Router Teacher to Guide MoE Student Learning](https://mlsanity.com/paper/2606.10369/) (2026)
- [Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation](https://mlsanity.com/paper/2608.10392/) (2026)
- [Pre-carved Niches: The Formation Dynamics of Modular Task Partitions in Early LLM Training](https://mlsanity.com/paper/2609.01170/) (2026)
- [CORTEX: Learning to Share and Specialize in Dense Language Models](https://mlsanity.com/paper/2609.34449/) (2026)

## Abstract (arXiv)

Modular Neural Networks (MNNs) demonstrate various advantages over monolithic models. Existing MNNs are generally textitexplicit: their modular architectures are pre-defined, with individual modules expected to implement distinct functions. Recent works reveal that there exists textitimplicit modularity in standard pre-trained transformers, namely textitEmergent Modularity. They indicate that such modular structures spontaneously exhibit during the early pre-training phase. Despite the benefits of modularity, most Language Models (LMs) are still treated as monolithic models in the pre-train and fine-tune paradigm, with their emergent modularity locked and underutilized. In this work, focusing on unlocking the emergent modularity in LMs, we showcase that standard LMs could be fine-tuned as their Mixture-of-Expert (MoEs) counterparts without introducing any extra parameters. Such MoEs are derived from emergent modularity and are referred to as Emergent MoEs (EMoE). Our experiments demonstrate that fine-tuning EMoE effectively improves downstream in-domain and out-of-domain generalization compared with vanilla fine-tuning. Our analysis and ablation studies further illustrate that it is robust to various configurations and can scale up to Large Language Models (i.e., Llama2-7B and Llama-30B). Code is available at https://github.com/qiuzh20/EMoE.