# InfMAE: A Foundation Model in the Infrared Modality arXiv: https://arxiv.org/abs/2402.00407 ยท HTML: https://mlsanity.com/paper/2402.00407/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 38 of its 60 references that are in the map. - First posted on arXiv 1 Feb 2024. ## Builds on (most cited first, 30 of 38) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017) - [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021) - [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014) - [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017) - [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021) - [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018) - [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021) - [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022) - [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017) - [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018) - [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023) - [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019) - [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023) - [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023) - [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021) - [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020) - [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022) - [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022) - [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022) - [PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers](https://mlsanity.com/paper/2111.12710/) (2021) - [Dense Nested Attention Network for Infrared Small Target Detection](https://mlsanity.com/paper/2106.00487/) (2021) - [Disentangled Non-Local Neural Networks](https://mlsanity.com/paper/2006.06668/) (2020) - [Benchmarking Detection Transfer Learning with Vision Transformers](https://mlsanity.com/paper/2111.11429/) (2021) ## Built on by (most cited first, 11 of 11) - [UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation](https://mlsanity.com/paper/2502.02257/) (2025) - [CM-Diff: A Single Generative Network for Bidirectional Cross-Modality Translation Diffusion Model Between Infrared and Visible Images](https://mlsanity.com/paper/2503.09514/) (2025) - [IV-tuning: Parameter-Efficient Transfer Learning for Infrared-Visible Tasks](https://mlsanity.com/paper/2412.16654/) (2024) - [Weakly-supervised Contrastive Learning with Quantity Prompts for Moving Infrared Small Target Detection](https://mlsanity.com/paper/2507.02454/) (2025) - [Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling](https://mlsanity.com/paper/2603.25170/) (2026) - [IVGF: The Fusion-Guided Infrared and Visible General Framework](https://mlsanity.com/paper/2409.00973/) (2024) - [UNIV: Unified Foundation Model for Infrared and Visible Modalities](https://mlsanity.com/paper/2509.15642/) (2025) - [DuGI-MAE: Improving Infrared Mask Autoencoders via Dual-Domain Guidance](https://mlsanity.com/paper/2512.04511/) (2025) - [SpectraDINO: Modality-Conditioned Adaptation of RGB Vision Foundation Models Across Infrared Bands](https://mlsanity.com/paper/2605.02258/) (2026) - [Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training](https://mlsanity.com/paper/2607.20238/) (2026) - [Representation Is Not Enough: Body-Localized Thermal Evidence for Contactless Stress and Craving Sensing in Opioid Use Disorder](https://mlsanity.com/paper/2608.16087/) (2026) ## Abstract (arXiv) In recent years, the foundation models have swept the computer vision field and facilitated the development of various tasks within different modalities. However, it remains an open question on how to design an infrared foundation model. In this paper, we propose InfMAE, a foundation model in infrared modality. We release an infrared dataset, called Inf30 to address the problem of lacking large-scale data for self-supervised learning in the infrared vision community. Besides, we design an information-aware masking strategy, which is suitable for infrared images. This masking strategy allows for a greater emphasis on the regions with richer information in infrared images during the self-supervised learning process, which is conducive to learning the generalized representation. In addition, we adopt a multi-scale encoder to enhance the performance of the pre-trained encoders in downstream tasks. Finally, based on the fact that infrared images do not have a lot of details and texture information, we design an infrared decoder module, which further improves the performance of downstream tasks. Extensive experiments show that our proposed method InfMAE outperforms other supervised methods and self-supervised learning methods in three downstream tasks.