mlsanityData to 7 Oct 2026

Accelerating Convolutional Neural Networks via Activation Map Compression

arXiv 1812.04056 ↗First posted 10 Dec 2018cs.CV cs.AI cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

The deep learning revolution brought us an extensive array of neural network architectures that achieve state-of-the-art performance in a wide variety of Computer Vision tasks including among others, classification, detection and segmentation. In parallel, we have also been observing an unprecedented demand in computational and memory requirements, rendering the efficient use of neural networks in low-powered devices virtually unattainable. Towards this end, we propose a three-stage compression and acceleration pipeline that sparsifies, quantizes and entropy encodes activation maps of Convolutional Neural Networks. Sparsification increases the representational power of activation maps leading to both acceleration of inference and higher model accuracy. Inception-V3 and MobileNet-V1 can be accelerated by as much as 1.6× with an increase in accuracy of 0.38% and 0.54% on the ImageNet and CIFAR-10 datasets respectively. Quantizing and entropy coding the sparser activation maps lead to higher compression over the baseline, reducing the memory cost of the network execution. Inception-V3 and MobileNet-V1 activation maps, quantized to 16 bits, are compressed by as much as 6× with an increase in accuracy of 0.36% and 0.55% respectively.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 7; 2024: 10; 2025: 4; 2026: 1.

34 papers.

Sort:

By area:

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 121
  4. 2024ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Modelscited by 43
  5. 2022NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networkscited by 38
  6. 2022Fine-tuning Language Models over Slow Networks using Activation Compression with Guaranteescited by 29
  7. 2025Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representationcited by 25
  8. 2021DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Devicecited by 17
  9. 2023Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversioncited by 15
  10. 2023Emergent representations in networks trained with the Forward-Forward algorithmcited by 11
  11. 2023Accelerating Deep Neural Networks via Semi-Structured Activation Sparsitycited by 10
  12. 2023Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devicescited by 7
22 more
  1. 2024The Impact of Uniform Inputs on Activation Sparsity and Energy-Latency Attacks in Computer Visioncited by 7
  2. 2024Event-based Optical Flow on Neuromorphic Processor: ANN vs. SNN Comparison based on Activation Sparsificationcited by 7
  3. 2024Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Trainingcited by 7
  4. 2022Survey: Exploiting Data Redundancy for Optimization of Deep Learningcited by 6
  5. 2024Activation Map Compression through Tensor Decomposition for Deep Learningcited by 6
  6. 2024Deep Neural Network Initialization with Sparsity Inducing Activationscited by 5
  7. 2022Optimizing the Consumption of Spiking Neural Networks with Activity Regularizationcited by 4
  8. 2022DIVISION: Memory Efficient Training via Dual Activation Precisioncited by 4
  9. 2023SPADE: Sparse Pillar-based 3D Object Detection Accelerator for Autonomous Drivingcited by 4
  10. 2021High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networkscited by 3
  11. 2025Sparse Convolutional Recurrent Learning for Efficient Event-based Neuromorphic Object Detectioncited by 3
  12. 2021Training for temporal sparsity in deep neural networks, application in video processingcited by 2
  13. 2022Improved Projection Learning for Lower Dimensional Feature Mapscited by 2
  14. 2025TAH-QUANT: Effective Activation Quantization in Pipeline Parallelism over Slow Networkcited by 2
  15. 2023Bandwidth-efficient Inference for Neural Image Compressioncited by 1
  16. 2022An Efficient Split Fine-tuning Framework for Edge and Cloud Collaborative Learningcited by 0
  17. 2022FlatENN: Train Flat for Enhanced Fault Tolerance of Quantized Deep Neural Networkscited by 0
  18. 2024Dual sparse training framework: inducing activation map sparsity via Transformed ℓ1 regularizationcited by 0
  19. 2024Memory Faults in Activation-sparse Quantized Deep Neural Networks: Analysis and Mitigation using Sharpness-aware Trainingcited by 0
  20. 2024MOYU: A Theoretical Study on Massive Over-activation Yielded Uplifts in LLMscited by 0
  21. 2025Memory Constrained Dynamic Subnetwork Update for Transfer Learningcited by 0
  22. 2026Towards the Connection between Activation Sparsity and Flat Minimacited by 0