mlsanityData to 7 Oct 2026

Rewrite the Stars

arXiv 2403.19967 ↗First posted 29 Mar 2024cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

Recent studies have drawn attention to the untapped potential of the "star operation" (element-wise multiplication) in network design. While intuitive explanations abound, the foundational rationale behind its application remains largely unexplored. Our study attempts to reveal the star operation's ability to map inputs into high-dimensional, non-linear feature spaces -- akin to kernel tricks -- without widening the network. We further introduce StarNet, a simple yet powerful prototype, demonstrating impressive performance and low latency under compact network structure and efficient budget. Like stars in the sky, the star operation appears unremarkable but holds a vast universe of potential. Our work encourages further exploration across tasks, with codes available at https://github.com/ma-xu/Rewrite-the-Stars.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 50 of its 63 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

50 of 63 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  10. 2017Squeeze-and-Excitation Networkscited by 4,763
  11. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
38 more
  1. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  2. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  3. 2022A ConvNet for the 2020scited by 3,545
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  7. 2020GLU Variants Improve Transformercited by 1,724
  8. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  9. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  10. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  11. 2019When Does Label Smoothing Help?cited by 985
  12. 2016Deep Networks with Stochastic Depthcited by 973
  13. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  14. 2021Going deeper with Image Transformerscited by 759
  15. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  16. 2019GhostNet: More Features from Cheap Operationscited by 410
  17. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  18. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  19. 2019Dynamic Convolution: Attention over Convolution Kernelscited by 355
  20. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  21. 2022Visual Attention Networkcited by 210
  22. 2022Focal Modulation Networkscited by 186
  23. 2021Dynamic Head: Unifying Object Detection Heads with Attentionscited by 185
  24. 2018Gather-Excite: Exploiting Feature Context in Convolutional Neural Networkscited by 138
  25. 2023Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networkscited by 129
  26. 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
  27. 2022MobileOne: An Improved One millisecond Mobile Backbonecited by 109
  28. 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
  29. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  30. 2022Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognitioncited by 81
  31. 2023FasterViT: Fast Vision Transformers with Hierarchical Attentioncited by 65
  32. 2023Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecturecited by 62
  33. 2020Dynamic ReLUcited by 55
  34. 2020Rethinking Bottleneck Structure for Efficient Mobile Network Designcited by 54
  35. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 52
  36. 2023Scale-Aware Modulation Meet Transformercited by 49
  37. 2023Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognitioncited by 18
  38. 2023SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 33; 2026: 9.

55 papers.

Sort:

By area:

  1. 2024LightStereo: Channel Boost Is All You Need for Efficient 2D Cost Aggregationcited by 28
  2. 2025OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernelscited by 24
  3. 2025LSNet: See Large, Focus Smallcited by 15
  4. 2025MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devicescited by 12
  5. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  6. 2025NTIRE 2025 Challenge on RAW Image Restoration and Super-Resolutioncited by 9
  7. 2024DA-Flow: Dual Attention Normalizing Flow for Skeleton-based Video Anomaly Detectioncited by 7
  8. 2024MAT: Multi-Range Attention Transformer for Efficient Image Super-Resolutioncited by 5
  9. 2024RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterizationcited by 4
  10. 2025Ultra-Low Bitrate Perceptual Image Compression with Shallow Encodercited by 4
  11. 2025S2AFormer: Strip Self-Attention for Efficient Vision Transformercited by 3
  12. 2025UniConvNet: Expanding Effective Receptive Field while Maintaining Asymptotically Gaussian Distribution for ConvNets of Any Scalecited by 3
38 more
  1. 2024Infinite-Dimensional Feature Interactioncited by 2
  2. 2024Temporal Reversal Regularization for Spiking Neural Networks: Hybrid Spatio-Temporal Invariance for Generalizationcited by 2
  3. 2025Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentationcited by 2
  4. 2025Scale-Aware Pre-Training for Human-Centric Visual Perception: Enabling Lightweight and Generalizable Modelscited by 2
  5. 2025Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Modelcited by 2
  6. 2024HASN: Hybrid Attention Separable Network for Efficient Image Super-resolutioncited by 1
  7. 2024RSNet: A Light Framework for The Detection of SAR Ship Detectioncited by 1
  8. 2024CrackESS: A Self-Prompting Crack Segmentation System for Edge Devicescited by 1
  9. 2024Towards Context-aware Convolutional Network for Image Restorationcited by 1
  10. 2025Multi-View Factorizing and Disentangling: A Novel Framework for Incomplete Multi-View Multi-Label Classificationcited by 1
  11. 2025LightFormer: A lightweight and efficient decoder for remote sensing image segmentationcited by 1
  12. 2025TableCenterNet: A one-stage network for table structure recognitioncited by 1
  13. 2025FTCFormer: Fuzzy Token Clustering Transformer for Image Classificationcited by 1
  14. 2025E-ConvNeXt: A Lightweight and Efficient ConvNeXt Variant with Cross-Stage Partial Connectionscited by 1
  15. 2024Monocular Obstacle Avoidance Based on Inverse PPO for Fixed-wing UAVscited by 0
  16. 2024Learning Dynamic Local Context Representations for Infrared Small Target Detectioncited by 0
  17. 2024RecConv: Efficient Recursive Convolutions for Multi-Frequency Representationscited by 0
  18. 2025A Separable Self-attention Inspired by the State Space Model for Computer Visioncited by 0
  19. 2025UL-UNAS: Ultra-Lightweight U-Nets for Real-Time Speech Enhancement via Network Architecture Searchcited by 0
  20. 2025Rapfi: Distilling Efficient Neural Network for the Game of Gomokucited by 0
  21. 2025RSRWKV: A Linear-Complexity 2D Attention Mechanism for Efficient Remote Sensing Vision Taskcited by 0
  22. 2025GmNet: Revisiting Gating Mechanisms From A Frequency Viewcited by 0
  23. 2025LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detectioncited by 0
  24. 2025CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Trackingcited by 0
  25. 2025Expert-Like Reparameterization of Heterogeneous Pyramid Receptive Fields in Efficient CNNs for Fair Medical Image Classificationcited by 0
  26. 2025Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Productscited by 0
  27. 2025DeepTraverse: A Depth-First Search Inspired Network for Algorithmic Visual Understandingcited by 0
  28. 2025Efficient Star Distillation Attention Network for Lightweight Image Super-Resolutioncited by 0
  29. 2025Exploration of Low-Cost but Accurate Radar-Based Human Motion Direction Determinationcited by 0
  30. 2025Rate-distortion Optimized Point Cloud Preprocessing for Geometry-based Point Cloud Compressioncited by 0
  31. 2025A Lightweight Group Multiscale Bidirectional Interactive Network for Real-Time Steel Surface Defect Detectioncited by 0
  32. 2025Designing Practical Models for Isolated Word Visual Speech Recognitioncited by 0
  33. 2025Advancing time series completion via RFAMoE and MDFFcited by 0
  34. 2025A Lightweight Multi-Scale Attention Framework for Real-Time Spinal Endoscopic Instance Segmentationcited by 0
  35. 2026Human-inspired Global-to-Parallel Multi-scale Encoding for Lightweight Vision Modelscited by 0
  36. 2026No Labels, No Look-Ahead: Unsupervised Online Video Stabilization with Classical Priorscited by 0
  37. 2026Universal Pansharpening Modelcited by 0
  38. 2026FSCE: A Target-Aware Frequency-Spatial Collaborative Enhancement Framework for Noise-Resilient SAR ATRcited by 0

50 most cited of 55 All 55 (JSON)