# CompactifAI: Extreme Compression of Large Language Models using Quantum-Inspired Tensor Networks

arXiv: https://arxiv.org/abs/2401.14109 · HTML: https://mlsanity.com/paper/2401.14109/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Tensor Networks Meet Neural Networks: A Survey and Future Perspectives](https://mlsanity.com/paper/2302.09019/) (2023)
- [Quantum-Inspired Tensor Neural Networks for Partial Differential Equations](https://mlsanity.com/paper/2208.02235/) (2022)

## Built on by (most cited first, 30 of 32)

- [An overview of model uncertainty and variability in LLM-based sentiment analysis. Challenges, mitigation strategies and the role of explainability](https://mlsanity.com/paper/2504.04462/) (2025)
- [Quantum Large Language Models via Tensor Network Disentanglers](https://mlsanity.com/paper/2410.17397/) (2024)
- [Quantum Natural Language Processing](https://mlsanity.com/paper/2403.19758/) (2024)
- [Quantum-Enhanced LLM Efficient Fine Tuning](https://mlsanity.com/paper/2503.12790/) (2025)
- [QuIC: Quantum-Inspired Compound Adapters for Parameter Efficient Fine-Tuning](https://mlsanity.com/paper/2502.06916/) (2025)
- [Tensor Network for Anomaly Detection in the Latent Space of Proton Collision Events at the LHC](https://mlsanity.com/paper/2506.00102/) (2025)
- [How to Square Tensor Networks and Circuits Without Squaring Them](https://mlsanity.com/paper/2512.17090/) (2025)
- [Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks](https://mlsanity.com/paper/2505.20132/) (2025)
- [Quantum Algorithms for Compositional Text Processing](https://mlsanity.com/paper/2408.06061/) (2024)
- [Quantum-enhanced unsupervised image segmentation for medical images analysis](https://mlsanity.com/paper/2411.15086/) (2024)
- [A Practical Tensor-Network Compression Pipeline for Production-Scale Large Language Models](https://mlsanity.com/paper/2602.01613/) (2026)
- [Compressing Transformer Language Models via Matrix Product Operator Decomposition: A Case Study on PicoGPT](https://mlsanity.com/paper/2603.28534/) (2026)
- [Tensor network compressibility of convolutional models](https://mlsanity.com/paper/2403.14379/) (2024)
- [Quantum-inspired Techniques in Tensor Networks for Industrial Contexts](https://mlsanity.com/paper/2404.11277/) (2024)
- [Accuracy and Consumption analysis from a compressed model by CompactifAI from Multiverse Computing](https://mlsanity.com/paper/2507.08836/) (2025)
- [No-Free-Lunch Theories for Tensor-Network Machine Learning Models](https://mlsanity.com/paper/2412.05674/) (2024)
- [Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability](https://mlsanity.com/paper/2608.30505/) (2026)
- [T'ecnicas Quantum-Inspired en Tensor Networks para Contextos Industriales](https://mlsanity.com/paper/2404.17645/) (2024)
- [QIANets: Quantum-Integrated Adaptive Networks for Reduced Latency and Improved Inference Times in CNN Models](https://mlsanity.com/paper/2410.10318/) (2024)
- [Explaining Anomalies with Tensor Networks](https://mlsanity.com/paper/2505.03911/) (2025)
- [A tensor network approach for chaotic time series prediction](https://mlsanity.com/paper/2505.17740/) (2025)
- [epiGPTope: A machine learning-based epitope generator and classifier](https://mlsanity.com/paper/2509.03351/) (2025)
- [Private and interpretable clinical prediction with quantum-inspired tensor train models](https://mlsanity.com/paper/2602.06110/) (2026)
- [Weight Clustering Reveals the Importance of Relative Rank in Large Language Models](https://mlsanity.com/paper/2603.17917/) (2026)
- [Quantum-inspired tensor networks in machine learning models](https://mlsanity.com/paper/2604.14287/) (2026)
- [Entanglement is Half the Story: Post-Selection vs. Partial Traces](https://mlsanity.com/paper/2605.02385/) (2026)
- [Fast Tensorization of Neural Networks via Slice-wise Feature Distillation](https://mlsanity.com/paper/2605.19842/) (2026)
- [Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations](https://mlsanity.com/paper/2607.28319/) (2026)
- [Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs](https://mlsanity.com/paper/2608.20953/) (2026)
- [Emergence of Fibrations, Compression, and Symmetry Breaking in Artificial Neural Networks](https://mlsanity.com/paper/2609.01768/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) such as ChatGPT and LlaMA are advancing rapidly in generative Artificial Intelligence (AI), but their immense size poses significant challenges, such as huge training and inference costs, substantial energy demands, and limitations for on-site deployment. Traditional compression methods such as pruning, distillation, and low-rank approximation focus on reducing the effective number of neurons in the network, while quantization focuses on reducing the numerical precision of individual weights to reduce the model size while keeping the number of neurons fixed. While these compression methods have been relatively successful in practice, there is no compelling reason to believe that truncating the number of neurons is an optimal strategy. In this context, this paper introduces CompactifAI, an innovative LLM compression approach using quantum-inspired Tensor Networks that focuses on the model's correlation space instead, allowing for a more controlled, refined and interpretable model compression. Our method is versatile and can be implemented with - or on top of - other compression techniques. As a benchmark, we demonstrate that a combination of CompactifAI with quantization allows to reduce a 93% the memory size of LlaMA 7B, reducing also 70% the number of parameters, accelerating 50% the training and 25% the inference times of the model, and just with a small accuracy drop of 2% - 3%, going much beyond of what is achievable today by other compression techniques. Our methods also allow to perform a refined layer sensitivity profiling, showing that deeper layers tend to be more suitable for tensor network compression, which is compatible with recent observations on the ineffectiveness of those layers for LLM performance. Our results imply that standard LLMs are, in fact, heavily overparametrized, and do not need to be large at all.