# Rewrite the Stars

arXiv: https://arxiv.org/abs/2403.19967 · HTML: https://mlsanity.com/paper/2403.19967/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 63 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)
- [Hyena Hierarchy: Towards Larger Convolutional Language Models](https://mlsanity.com/paper/2302.10866/) (2023)
- [High-Performance Large-Scale Image Recognition Without Normalization](https://mlsanity.com/paper/2102.06171/) (2021)

## Built on by (most cited first, 30 of 55)

- [LightStereo: Channel Boost Is All You Need for Efficient 2D Cost Aggregation](https://mlsanity.com/paper/2406.19833/) (2024)
- [OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels](https://mlsanity.com/paper/2502.20087/) (2025)
- [LSNet: See Large, Focus Small](https://mlsanity.com/paper/2503.23135/) (2025)
- [MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devices](https://mlsanity.com/paper/2507.01838/) (2025)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [NTIRE 2025 Challenge on RAW Image Restoration and Super-Resolution](https://mlsanity.com/paper/2506.02197/) (2025)
- [DA-Flow: Dual Attention Normalizing Flow for Skeleton-based Video Anomaly Detection](https://mlsanity.com/paper/2406.02976/) (2024)
- [Ultra-Low Bitrate Perceptual Image Compression with Shallow Encoder](https://mlsanity.com/paper/2512.12229/) (2025)
- [MAT: Multi-Range Attention Transformer for Efficient Image Super-Resolution](https://mlsanity.com/paper/2411.17214/) (2024)
- [RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization](https://mlsanity.com/paper/2406.16004/) (2024)
- [S2AFormer: Strip Self-Attention for Efficient Vision Transformer](https://mlsanity.com/paper/2505.22195/) (2025)
- [UniConvNet: Expanding Effective Receptive Field while Maintaining Asymptotically Gaussian Distribution for ConvNets of Any Scale](https://mlsanity.com/paper/2508.09000/) (2025)
- [Infinite-Dimensional Feature Interaction](https://mlsanity.com/paper/2405.13972/) (2024)
- [Temporal Reversal Regularization for Spiking Neural Networks: Hybrid Spatio-Temporal Invariance for Generalization](https://mlsanity.com/paper/2408.09108/) (2024)
- [Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2501.17642/) (2025)
- [Scale-Aware Pre-Training for Human-Centric Visual Perception: Enabling Lightweight and Generalizable Models](https://mlsanity.com/paper/2503.08201/) (2025)
- [Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model](https://mlsanity.com/paper/2510.11509/) (2025)
- [HASN: Hybrid Attention Separable Network for Efficient Image Super-resolution](https://mlsanity.com/paper/2410.09844/) (2024)
- [RSNet: A Light Framework for The Detection of SAR Ship Detection](https://mlsanity.com/paper/2410.23073/) (2024)
- [CrackESS: A Self-Prompting Crack Segmentation System for Edge Devices](https://mlsanity.com/paper/2412.07205/) (2024)
- [Towards Context-aware Convolutional Network for Image Restoration](https://mlsanity.com/paper/2412.11008/) (2024)
- [Multi-View Factorizing and Disentangling: A Novel Framework for Incomplete Multi-View Multi-Label Classification](https://mlsanity.com/paper/2501.06524/) (2025)
- [LightFormer: A lightweight and efficient decoder for remote sensing image segmentation](https://mlsanity.com/paper/2504.10834/) (2025)
- [TableCenterNet: A one-stage network for table structure recognition](https://mlsanity.com/paper/2504.17522/) (2025)
- [CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Tracking](https://mlsanity.com/paper/2505.05936/) (2025)
- [FTCFormer: Fuzzy Token Clustering Transformer for Image Classification](https://mlsanity.com/paper/2507.10283/) (2025)
- [E-ConvNeXt: A Lightweight and Efficient ConvNeXt Variant with Cross-Stage Partial Connections](https://mlsanity.com/paper/2508.20955/) (2025)
- [Monocular Obstacle Avoidance Based on Inverse PPO for Fixed-wing UAVs](https://mlsanity.com/paper/2411.18009/) (2024)
- [Learning Dynamic Local Context Representations for Infrared Small Target Detection](https://mlsanity.com/paper/2412.17401/) (2024)
- [RecConv: Efficient Recursive Convolutions for Multi-Frequency Representations](https://mlsanity.com/paper/2412.19628/) (2024)

## Abstract (arXiv)

Recent studies have drawn attention to the untapped potential of the "star operation" (element-wise multiplication) in network design. While intuitive explanations abound, the foundational rationale behind its application remains largely unexplored. Our study attempts to reveal the star operation's ability to map inputs into high-dimensional, non-linear feature spaces -- akin to kernel tricks -- without widening the network. We further introduce StarNet, a simple yet powerful prototype, demonstrating impressive performance and low latency under compact network structure and efficient budget. Like stars in the sky, the star operation appears unremarkable but holds a vast universe of potential. Our work encourages further exploration across tasks, with codes available at https://github.com/ma-xu/Rewrite-the-Stars.