mlsanityData to 7 Oct 2026

EinSort: Sorting is All We Need for Tensorizing LLM

arXiv 2606.08565 ↗First posted 7 Jun 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Tensor networks provide efficient representations for compressing large neural networks. By carefully designing shapes and topologies, they can significantly reduce memory and computational costs. However, identifying implicit low-rank structures in large foundation models remains challenging due to their enormous scale and un-structured weight distributions. We propose an adaptive tensorization method that discovers inherent low-rank structure in a target tensor by index ordering. Experiments on weight and KV-cache compression demonstrate improved reconstruction quality compared to baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 77 of its 97 references that are in the map. First posted on arXiv 7 Jun 2026.

Builds on

77 of 97 references.

References · 77

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2025Qwen3-VL Technical Reportcited by 2,534
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  11. 2019Towards VQA Models That Can Readcited by 1,696
  12. 2025Gemma 3 Technical Reportcited by 1,686
38 more
  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  2. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  4. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  6. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  7. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  8. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  9. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  10. 2024Phi-4 Technical Reportcited by 675
  11. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  12. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  13. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  14. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
  15. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  16. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 365
  17. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  18. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  19. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  20. 2023A Survey on Model Compression for Large Language Modelscited by 263
  21. 2015Tensorizing Neural Networkscited by 246
  22. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
  23. 2013Predicting Parameters in Deep Learningcited by 223
  24. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 223
  25. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  26. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
  27. 2022DaViT: Dual Attention Vision Transformerscited by 174
  28. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 172
  29. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 156
  30. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  31. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 130
  32. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
  33. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  34. 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
  35. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Modelcited by 104
  36. 2023One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuningcited by 84
  37. 2018Towards Quantum Machine Learning with Tensor Networkscited by 74
  38. 2020Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Networkcited by 74

50 most cited of 77 All 77 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking the Role of Tensor Decompositions in Post-Training LLM Compression
  2. 2025Domain-Aware Tensor Network Structure Search
  3. 2024Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection
  4. 2025Score-Based Model for Low-Rank Tensor Recovery
  5. 2023Deep Tensor Network
  6. 2024Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
  7. 2025LatentLLM: Attention-Aware Joint Tensor Compression
  8. 2026Algorithmic algorithm development with LLMs: A Case Study on LLM-Usage for Contraction Order Optimization in Tensor Networks