mlsanityData to 7 Oct 2026

Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions

arXiv 1711.08141 ↗First posted 22 Nov 2017cs.CVcited by 93

Not on the map. Why?

Abstract (arXiv)

Neural networks rely on convolutions to aggregate spatial information. However, spatial convolutions are expensive in terms of model size and computation, both of which grow quadratically with respect to kernel size. In this paper, we present a parameter-free, FLOP-free "shift" operation as an alternative to spatial convolutions. We fuse shifts and point-wise convolutions to construct end-to-end trainable shift-based modules, with a hyperparameter characterizing the tradeoff between accuracy and efficiency. To demonstrate the operation's efficacy, we replace ResNet's 3x3 convolutions with shift-based modules for improved CIFAR10 and CIFAR100 accuracy using 60% fewer parameters; we additionally demonstrate the operation's resilience to parameter reduction on ImageNet, outperforming ResNet family members. We finally show the shift operation's applicability across domains, achieving strong performance with fewer parameters on classification, face verification and style transfer.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 22; 2023: 15; 2024: 12; 2025: 10; 2026: 4.

93 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2022Efficient Long-Range Attention Network for Image Super-resolutioncited by 151
  3. 2021AS-MLP: An Axial Shifted MLP Architecture for Visioncited by 101
  4. 2021Space-time Mixing Attention for Video Transformercited by 84
  5. 2021S2-MLP: Spatial-Shift MLP Architecture for Visioncited by 83
  6. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  7. 2021Token Shift Transformer for Video Classificationcited by 61
  8. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  9. 2022Online Continual Learning for Embedded Devicescited by 48
  10. 2021Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directionscited by 41
  11. 2021FEAR: Fast, Efficient, Accurate and Robust Visual Trackercited by 38
  12. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
38 more
  1. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  2. 2021MixFaceNets: Extremely Efficient Face Recognition Networkscited by 36
  3. 2022NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxiescited by 36
  4. 2023EdgeFace: Efficient Face Recognition Model for Edge Devicescited by 33
  5. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  6. 2022When Shift Operation Meets Vision Transformer: An Extremely Simple Alternative to Attention Mechanismcited by 30
  7. 2021ASSANet: An Anisotropic Separable Set Abstraction for Efficient Point Cloud Representation Learningcited by 29
  8. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 27
  9. 2021Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusioncited by 25
  10. 2022DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networkscited by 19
  11. 2023LightCTS: A Lightweight Framework for Correlated Time Series Forecastingcited by 18
  12. 2023Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitivescited by 18
  13. 2024DyFADet: Dynamic Feature Aggregation for Temporal Action Detectioncited by 18
  14. 2021TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devicecited by 17
  15. 2021HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learningcited by 16
  16. 2021Rethinking Token-Mixing MLP for MLP-based Vision Backbonecited by 15
  17. 2023ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformercited by 15
  18. 2022GhostNets on Heterogeneous Devices via Cheap Operationscited by 14
  19. 2021InversionNet3D: Efficient and Scalable Learning for 3D Full Waveform Inversioncited by 12
  20. 2021MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurationscited by 12
  21. 2024Building a Multi-modal Spatiotemporal Expert for Zero-shot Action Recognition with CLIPcited by 9
  22. 2021GhostSR: Learning Ghost Features for Efficient Image Super-Resolutioncited by 8
  23. 2022ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networkscited by 8
  24. 2021Compacting Deep Neural Networks for Internet of Things: Methods and Applicationscited by 6
  25. 2022Towards Accurate Binary Neural Networks via Modeling Contextual Dependenciescited by 6
  26. 2022Linear Video Transformer with Feature Fixationcited by 6
  27. 2023Fully 1×1 Convolutional Network for Lightweight Image Super-Resolutioncited by 6
  28. 2024How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalancecited by 6
  29. 2025Efficient Dynamic Attention 3D Convolution for Hyperspectral Image Classificationcited by 6
  30. 2022Learning Features with Parameter-Free Layerscited by 5
  31. 2022Vision Transformer with Cross-attention by Temporal Shift for Efficient Action Recognitioncited by 5
  32. 2022EResFD: Rediscovery of the Effectiveness of Standard Convolution for Lightweight Face Detectioncited by 5
  33. 2022NASA: Neural Architecture Search and Acceleration for Hardware Inspired Hybrid Networkscited by 5
  34. 2024Efficient Low-Resolution Face Recognition via Bridge Distillationcited by 5
  35. 2025xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devicescited by 5
  36. 2025TS-SNN: Temporal Shift Module for Spiking Neural Networkscited by 5
  37. 2025UltraFast-LiNET: Light-weight multi-scale shift convolutional network for real-time low-light image enhancementcited by 5
  38. 2021RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detectioncited by 4

50 most cited of 93 All 93 (JSON)