mlsanityData to 7 Oct 2026

DCT Perceptron Layer: A Transform Domain Approach for Convolution Layer

arXiv 2211.08577 ↗First posted 15 Nov 2022cs.CV eess.IVcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel Discrete Cosine Transform (DCT)-based neural network layer which we call DCT-perceptron to replace the 3×3 Conv2D layers in the Residual neural Network (ResNet). Convolutional filtering operations are performed in the DCT domain using element-wise multiplications by taking advantage of the Fourier and DCT Convolution theorems. A trainable soft-thresholding layer is used as the nonlinearity in the DCT perceptron. Compared to ResNet's Conv2D layer which is spatial-agnostic and channel-specific, the proposed layer is location-specific and channel-specific. The DCT-perceptron layer reduces the number of parameters and multiplications significantly while maintaining comparable accuracy results of regular ResNets in CIFAR-10 and ImageNet-1K. Moreover, the DCT-perceptron layer can be inserted with a batch normalization layer before the global average pooling layer in the conventional ResNets as an additional layer to improve classification accuracy.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

22 of 44 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2022A ConvNet for the 2020scited by 3,545
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  8. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  9. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  10. 2017Residual Attention Network for Image Classificationcited by 385
  11. 2019Fast-SCNN: Fast Semantic Segmentation Networkcited by 124
  12. 2021Learning Auxiliary Monocular Contexts Helps Monocular 3D Object Detectioncited by 66
10 more
  1. 2019Multi-level Wavelet Convolutional Neural Networkscited by 59
  2. 2018Designing Adaptive Neural Networks for Energy-Constrained Image Classificationcited by 18
  3. 2020Harmonic Convolutional Networks based on Discrete Cosine Transformcited by 14
  4. 2021Fast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networkscited by 14
  5. 2022Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networkscited by 13
  6. 2018Compressing the Input for CNNs with the First-Order Scattering Transformcited by 11
  7. 2021Less is More: Accelerating Faster Neural Networks Straight from JPEGcited by 7
  8. 2019A Self-Attentive Emotion Recognition Networkcited by 5
  9. 2020Discrete Cosine Transform Based Causal Convolutional Neural Network for Drift Compensation in Chemical Sensorscited by 5
  10. 2020A Simulation Study to Evaluate the Performance of the Cauchy Proximal Operator in Despeckling SAR Images of the Sea Surfacecited by 4

Built on by

Cited by year

Citing papers per year: 2023: 3.

3 papers.

  1. 2023ADC/DAC-Free Analog Acceleration of Deep Neural Networks with Frequency Transformationcited by 3
  2. 2023A novel asymmetrical autoencoder with a sparsifying discrete cosine Stockwell transform layer for gearbox sensor data compressioncited by 3
  3. 2023Containing Analog Data Deluge at Edge through Frequency-Domain Compression in Collaborative Compute-in-Memory Networkscited by 0