MacFormer: Semantic Segmentation with Fine Object Boundaries
Not on the map. Why?
Abstract (arXiv)
Semantic segmentation involves assigning a specific category to each pixel in an image. While Vision Transformer-based models have made significant progress, current semantic segmentation methods often struggle with precise predictions in localized areas like object boundaries. To tackle this challenge, we introduce a new semantic segmentation architecture, ``MacFormer'', which features two key components. Firstly, using learnable agent tokens, a Mutual Agent Cross-Attention (MACA) mechanism effectively facilitates the bidirectional integration of features across encoder and decoder layers. This enables better preservation of low-level features, such as elementary edges, during decoding. Secondly, a Frequency Enhancement Module (FEM) in the decoder leverages high-frequency and low-frequency components to boost features in the frequency domain, benefiting object boundaries with minimal computational complexity increase. MacFormer is demonstrated to be compatible with various network architectures and outperforms existing methods in both accuracy and efficiency on benchmark datasets ADE20K and Cityscapes under different computational constraints.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 57 of its 69 references that are in the map. First posted on arXiv 11 Aug 2024.
Builds on
57 of 69 references.
References · 57
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2016Pyramid Scene Parsing Networkcited by 2,472
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
38 more
- 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
- 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
- 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
- 2024VMamba: Visual State Space Modelcited by 955
- 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
- 2021Multiscale Vision Transformerscited by 896
- 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
- 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
- 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
- 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
- 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
- 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
- 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
- 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
- 2018Context Encoding for Semantic Segmentationcited by 289
- 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
- 2021Global Filter Networks for Image Classificationcited by 257
- 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
- 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 235
- 2021Co-Scale Conv-Attentional Image Transformerscited by 224
- 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 204
- 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
- 2023Efficient and Explicit Modelling of Image Hierarchies for Image Restorationcited by 145
- 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
- 2019Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolutioncited by 140
- 2019Deep Semantic Segmentation of Natural and Medical Images: A Reviewcited by 124
- 2020Strip Pooling: Rethinking Spatial Pooling for Scene Parsingcited by 114
- 2023Agent Attention: On the Integration of Softmax and Linear Attentioncited by 90
- 2022TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentationcited by 82
- 2024ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictionscited by 65
- 2021Lite Vision Transformer with Enhanced Self-Attentioncited by 64
- 2021ConvMLP: Hierarchical Convolutional MLPs for Visioncited by 37
- 2024Frequency-Adaptive Dilated Convolution for Semantic Segmentationcited by 34
- 2022Deep Fourier Up-Samplingcited by 27
- 2023SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolutioncited by 27
- 2023SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentationcited by 22
- 2024Frequency Attention for Knowledge Distillationcited by 22
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 1; 2026: 1.
3 papers.