mlsanityData to 14 Oct 2026

MOGNET: A Mux-residual quantized Network leveraging Online-Generated weights

arXiv 2501.09531 ↗First posted 16 Jan 2025cs.LG cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper presents a compact model architecture called MOGNET, compatible with a resource-limited hardware. MOGNET uses a streamlined Convolutional factorization block based on a combination of 2 point-wise (1x1) convolutions with a group-wise convolution in-between. To further limit the overall model size and reduce the on-chip required memory, the second point-wise convolution's parameters are on-line generated by a Cellular Automaton structure. In addition, MOGNET enables the use of low-precision weights and activations, by taking advantage of a Multiplexer mechanism with a proper Bitshift rescaling for integrating residual paths without increasing the hardware-related complexity. To efficiently train this model we also introduce a novel weight ternarization method favoring the balance between quantized levels. Experimental results show that given tiny memory budget (sub-2Mb), MOGNET can achieve higher accuracy with a clear gap up to 1% at a similar or even lower model size compared to recent state-of-the-art methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 16 Jan 2025.

Builds on

21 of 28 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  6. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  12. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
9 more
  1. 2019Learned Step Size Quantizationcited by 628
  2. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  3. 2017Residual Attention Network for Image Classificationcited by 386
  4. 2016Ternary Weight Networkscited by 204
  5. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  6. 2019Scalable Model Compression by Entropy Penalized Reparameterizationcited by 31
  7. 2020Rethinking Depthwise Separable Convolutions: How Intra-Kernel Correlations Lead to Improved MobileNetscited by 28
  8. 2014Reservoir Computing using Cellular Automatacited by 8
  9. 2020Compressing Deep Convolutional Neural Networks by Stacking Low-dimensional Binary Convolution Filterscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization
  2. 2025BILLNET: A Binarized Conv3D-LSTM Network with Logic-gated residual architecture for hardware-efficient video inference
  3. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization
  4. 2026MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference
  5. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
  6. 2024Efficiera Residual Networks: Hardware-Friendly Fully Binary Weight with 2-bit Activation Model Achieves Practical ImageNet Accuracy
  7. 2023DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables
  8. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference