# Online Model Compression for Federated Learning with Large Models

arXiv: https://arxiv.org/abs/2205.03494 · HTML: https://mlsanity.com/paper/2205.03494/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 27 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 22 of 22)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Expanding the Reach of Federated Learning by Reducing Client Resource Requirements](https://mlsanity.com/paper/1812.07210/) (2018)
- [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://mlsanity.com/paper/1611.05128/) (2016)
- [Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling](https://mlsanity.com/paper/1902.08295/) (2019)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [A Better and Faster End-to-End Model for Streaming ASR](https://mlsanity.com/paper/2011.10798/) (2020)
- [Pareto-Optimal Quantized ResNet Is Mostly 4-bit](https://mlsanity.com/paper/2105.03536/) (2021)
- [4-bit Conformer with Native Quantization Aware Training for Speech Recognition](https://mlsanity.com/paper/2203.15952/) (2022)
- [Partial Variable Training for Efficient On-Device Federated Learning](https://mlsanity.com/paper/2110.05607/) (2021)
- [NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization](https://mlsanity.com/paper/2104.00031/) (2021)
- [Enabling On-Device Training of Speech Recognition Models with Federated Dropout](https://mlsanity.com/paper/2110.03634/) (2021)
- [Distributed Fixed Point Methods with Compressed Iterates](https://mlsanity.com/paper/1912.09925/) (2019)

## Built on by (most cited first, 7 of 7)

- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)
- [Vanishing Variance Problem in Fully Decentralized Neural-Network Systems](https://mlsanity.com/paper/2404.04616/) (2024)
- [Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach](https://mlsanity.com/paper/2604.21197/) (2026)
- [Federated Automatic Differentiation](https://mlsanity.com/paper/2301.07806/) (2023)
- [A Resource-Adaptive Approach for Federated Learning under Resource-Constrained Environments](https://mlsanity.com/paper/2406.13351/) (2024)
- [Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insights](https://mlsanity.com/paper/2509.05142/) (2025)
- [Federated Learning of Large ASR Models in the Real World](https://mlsanity.com/paper/2408.10443/) (2024)

## Abstract (arXiv)

This paper addresses the challenges of training large neural network models under federated learning settings: high on-device memory usage and communication cost. The proposed Online Model Compression (OMC) provides a framework that stores model parameters in a compressed format and decompresses them only when needed. We use quantization as the compression method in this paper and propose three methods, (1) using per-variable transformation, (2) weight matrices only quantization, and (3) partial parameter quantization, to minimize the impact on model accuracy. According to our experiments on two recent neural networks for speech recognition and two different datasets, OMC can reduce memory usage and communication cost of model parameters by up to 59% while attaining comparable accuracy and training speed when compared with full-precision training.