# A Model Compression Method with Matrix Product Operators for Speech Enhancement

arXiv: https://arxiv.org/abs/2010.04950 · HTML: https://mlsanity.com/paper/2010.04950/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study](https://mlsanity.com/paper/2307.08072/) (2023)
- [Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models](https://mlsanity.com/paper/2203.01104/) (2022)
- [Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators](https://mlsanity.com/paper/2106.02205/) (2021)
- [Tensor networks for interpretable and efficient quantum-inspired machine learning](https://mlsanity.com/paper/2311.11258/) (2023)
- [Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression](https://mlsanity.com/paper/2405.12591/) (2024)
- [Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation](https://mlsanity.com/paper/2411.06448/) (2024)
- [SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points](https://mlsanity.com/paper/2111.04436/) (2021)
- [Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture](https://mlsanity.com/paper/2303.16753/) (2023)
- [Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs](https://mlsanity.com/paper/2403.17607/) (2024)
- [Gaussian Universality and Its Breakdown in Tensor-Network Machine Learning](https://mlsanity.com/paper/2610.06080/) (2026)

## Abstract (arXiv)

The deep neural network (DNN) based speech enhancement approaches have achieved promising performance. However, the number of parameters involved in these methods is usually enormous for the real applications of speech enhancement on the device with the limited resources. This seriously restricts the applications. To deal with this issue, model compression techniques are being widely studied. In this paper, we propose a model compression method based on matrix product operators (MPO) to substantially reduce the number of parameters in DNN models for speech enhancement. In this method, the weight matrices in the linear transformations of neural network model are replaced by the MPO decomposition format before training. In experiment, this process is applied to the causal neural network models, such as the feedforward multilayer perceptron (MLP) and long short-term memory (LSTM) models. Both MLP and LSTM models with/without compression are then utilized to estimate the ideal ratio mask for monaural speech enhancement. The experimental results show that our proposed MPO-based method outperforms the widely-used pruning method for speech enhancement under various compression rates, and further improvement can be achieved with respect to low compression rates. Our proposal provides an effective model compression method for speech enhancement, especially in cloud-free application.