mlsanityData to 7 Oct 2026

Deformable Butterfly: A Highly Structured and Sparse Linear Transform

arXiv 2203.13556 ↗First posted 25 Mar 2022cs.CV cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce a new kind of linear transform named Deformable Butterfly (DeBut) that generalizes the conventional butterfly matrices and can be adapted to various input-output dimensions. It inherits the fine-to-coarse-grained learnable hierarchy of traditional butterflies and when deployed to neural networks, the prominent structures and sparsity in a DeBut layer constitutes a new way for network compression. We apply DeBut as a drop-in replacement of standard fully connected and convolutional layers, and demonstrate its superiority in homogenizing a neural network and rendering it favorable properties such as light weight and low inference complexity, without compromising accuracy. The natural complexity-accuracy tradeoff arising from the myriad deformations of a DeBut layer also opens up new rooms for analytical and practical research. The codes and Appendix are publicly available at: https://github.com/ruilin0212/DeBut.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 19 of its 44 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

19 of 44 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  11. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  12. 2015Tensorizing Neural Networkscited by 246
7 more
  1. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
  2. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  3. 2019Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizationscited by 73
  4. 2014Fastfood: Approximate Kernel Expansions in Loglinear Timecited by 72
  5. 2020Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Mapscited by 36
  6. 2014Deep Fried Convnetscited by 21
  7. 2021Exploiting Elasticity in Tensor Ranks for Compressing Neural Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 1; 2025: 1.

8 papers.

  1. 2022Monarch: Expressive Structured Matrices for Efficient and Accurate Trainingcited by 100
  2. 2023Simple Hardware-Efficient Long Convolutions for Sequence Modelingcited by 62
  3. 2023Can sparsity improve the privacy of neural networks?cited by 1
  4. 2023Sparsity in neural networks can improve their privacycited by 1
  5. 2023Lite it fly: An All-Deformable-Butterfly Networkcited by 0
  6. 2023Dimension Mixer: Group Mixing of Input Dimensions for Efficient Function Approximationcited by 0
  7. 2024Fast Inference with Kronecker-Sparse Matricescited by 0
  8. 2025Efficient Tensor Completion Algorithms for Highly Oscillatory Operatorscited by 0