# DenoMAE2.0: Improving Denoising Masked Autoencoders by Classifying Local Patches

arXiv: https://arxiv.org/abs/2502.18202 · HTML: https://mlsanity.com/paper/2502.18202/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Over the Air Deep Learning Based Radio Signal Classification](https://mlsanity.com/paper/1712.04578/) (2017)
- [Semi-supervised Vision Transformers at Scale](https://mlsanity.com/paper/2208.05688/) (2022)
- [StableMask: Refining Causal Masking in Decoder-only Transformer](https://mlsanity.com/paper/2402.04779/) (2024)
- [An Efficient Deep Learning Model for Automatic Modulation Recognition Based on Parameter Estimation and Transformation](https://mlsanity.com/paper/2110.04980/) (2021)
- [Understanding Masked Autoencoders From a Local Contrastive Perspective](https://mlsanity.com/paper/2310.01994/) (2023)
- [Semi-Supervised Learning via Cross-Prediction-Powered Inference for Wireless Systems](https://mlsanity.com/paper/2405.15415/) (2024)
- [Transformers are Provably Optimal In-context Estimators for Wireless Communications](https://mlsanity.com/paper/2311.00226/) (2023)
- [NMformer: A Transformer for Noisy Modulation Classification in Wireless Communication](https://mlsanity.com/paper/2411.02428/) (2024)
- [Revisiting Pretraining for Semi-Supervised Learning in the Low-Label Regime](https://mlsanity.com/paper/2205.03001/) (2022)
- [Learning Constellation Map with Deep CNN for Accurate Modulation Recognition](https://mlsanity.com/paper/2009.02026/) (2020)
- [Deep Learning Radio Frequency Signal Classification with Hybrid Images](https://mlsanity.com/paper/2105.09063/) (2021)
- [SigT: An Efficient End-to-End MIMO-OFDM Receiver Framework Based on Transformer](https://mlsanity.com/paper/2211.09712/) (2022)
- [Transformer Masked Autoencoders for Next-Generation Wireless Communications: Architecture and Opportunities](https://mlsanity.com/paper/2401.06274/) (2024)
- [SSwsrNet: A Semi-Supervised Few-Shot Learning Framework for Wireless Signal Recognition](https://mlsanity.com/paper/2404.02467/) (2024)
- [DenoMAE: A Multimodal Autoencoder for Denoising Modulation Signals](https://mlsanity.com/paper/2501.11538/) (2025)
- [Deep Learning Frameworks for Cognitive Radio Networks: Review and Open Research Challenges](https://mlsanity.com/paper/2410.23949/) (2024)
- [MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning](https://mlsanity.com/paper/2405.06227/) (2024)

## Built on by (most cited first, 3 of 3)

- [Plug-and-Play AMC: Context Is King in Training-Free, Open-Set Modulation with LLMs](https://mlsanity.com/paper/2505.03112/) (2025)
- [A Foundation Model for DAS Signal Recognition and Visual Prompt Tuning of the Pre-trained Model for Downstream Tasks](https://mlsanity.com/paper/2508.04316/) (2025)
- [Large Language Models Can Perform Automatic Modulation Classification via Discretized Self-supervised Candidate Retrieval](https://mlsanity.com/paper/2510.00316/) (2025)

## Abstract (arXiv)

We introduce DenoMAE2.0, an enhanced denoising masked autoencoder that integrates a local patch classification objective alongside traditional reconstruction loss to improve representation learning and robustness. Unlike conventional Masked Autoencoders (MAE), which focus solely on reconstructing missing inputs, DenoMAE2.0 introduces position-aware classification of unmasked patches, enabling the model to capture fine-grained local features while maintaining global coherence. This dual-objective approach is particularly beneficial in semi-supervised learning for wireless communication, where high noise levels and data scarcity pose significant challenges. We conduct extensive experiments on modulation signal classification across a wide range of signal-to-noise ratios (SNRs), from extremely low to moderately high conditions and in a low data regime. Our results demonstrate that DenoMAE2.0 surpasses its predecessor, Deno-MAE, and other baselines in both denoising quality and downstream classification accuracy. DenoMAE2.0 achieves a 1.1% improvement over DenoMAE on our dataset and 11.83%, 16.55% significant improved accuracy gains on the RadioML benchmark, over DenoMAE, for constellation diagram classification of modulation signals.