mlsanityData to 7 Oct 2026

AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge

arXiv 2104.14623 ↗First posted 29 Apr 2021cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In this study, we introduce AttendSeg, a low-precision, highly compact deep neural network tailored for on-device semantic segmentation. AttendSeg possesses a self-attention network architecture comprising of light-weight attention condensers for improved spatial-channel selective attention at a very low complexity. The unique macro-architecture and micro-architecture design properties of AttendSeg strike a strong balance between representational power and efficiency, achieved via a machine-driven design exploration strategy tailored specifically for the task at hand. Experimental results demonstrated that the proposed AttendSeg can achieve segmentation accuracy comparable to much larger deep neural networks with greater complexity while possessing a significantly lower architecture and computational complexity (requiring as much as >27x fewer MACs, >72x fewer parameters, and >288x lower weight memory requirements), making it well-suited for TinyML applications on the edge.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 29 Apr 2021.

Builds on

37 of 47 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  10. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  11. 2016Pyramid Scene Parsing Networkcited by 2,472
  12. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
25 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  5. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  6. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  7. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  8. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  9. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 540
  10. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 463
  11. 2017Understanding Convolution for Semantic Segmentationcited by 168
  12. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  13. 2020Learning Dynamic Routing for Semantic Segmentationcited by 79
  14. 2018NetScore: Towards Universal Metrics for Large-scale Performance Analysis of Deep Neural Networks for Practical On-Device Edge Usagecited by 30
  15. 2018FermiNets: Learning generative machines to generate efficient neural networks via generative synthesiscited by 21
  16. 2018Task Transfer by Preference-Based Cost Learningcited by 18
  17. 2020TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devicescited by 16
  18. 2017ProjectionNet: Learning Efficient On-Device Deep Networks Using Neural Projectionscited by 11
  19. 2020AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condenserscited by 10
  20. 2018Tiny SSD: A Tiny Single-shot Detection Deep Convolutional Neural Network for Real-time Embedded Object Detectioncited by 8
  21. 2019EdgeSegNet: A Compact Network for Semantic Segmentationcited by 4
  22. 2016Deep Learning with Darwin: Evolutionary Synthesis of Deep Neural Networkscited by 3
  23. 2017SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesiscited by 3
  24. 2017Two-Bit Networks for Deep Learning on Resource-Constrained Embedded Devicescited by 1
  25. 2019Implications of Computer Vision Driven Assistive Technologies Towards Individuals with Visual Impairmentcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1.

4 papers.

  1. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  2. 2022Faster Attention Is What You Need: A Fast Self-Attention Neural Network Backbone Architecture for the Edge via Double-Condensing Attention Condenserscited by 5
  3. 2022L^3U-net: Low-Latency Lightweight U-net Based Image Segmentation Model for Parallel CNN Processorscited by 2
  4. 2023FMAS: Fast Multi-Objective SuperNet Architecture Search for Semantic Segmentationcited by 0