# Atomizer: Generalizing to new modalities by breaking satellite images down to a set of scalars

arXiv: https://arxiv.org/abs/2506.13542 · HTML: https://mlsanity.com/paper/2506.13542/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [Lightweight, Pre-trained Transformers for Remote Sensing Timeseries](https://mlsanity.com/paper/2304.14065/) (2023)
- [reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysis](https://mlsanity.com/paper/2407.03653/) (2024)
- [FlexiMo: A Flexible Remote Sensing Foundation Model](https://mlsanity.com/paper/2503.23844/) (2025)
- [SenPa-MAE: Sensor Parameter Aware Masked Autoencoder for Multi-Satellite Self-Supervised Pretraining](https://mlsanity.com/paper/2408.11000/) (2024)
- [Self-Attentive Pooling for Efficient Deep Learning](https://mlsanity.com/paper/2209.07659/) (2022)
- [Planted: a dataset for planted forest identification from multi-satellite time series](https://mlsanity.com/paper/2406.18554/) (2024)

## Built on by (most cited first, 1 of 1)

- [UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation](https://mlsanity.com/paper/2606.23503/) (2026)

## Abstract (arXiv)

The growing number of Earth observation satellites has led to increasingly diverse remote sensing data, with varying spatial, spectral, and temporal configurations. Most existing models rely on fixed input formats and modality-specific encoders, which require retraining when new configurations are introduced, limiting their ability to generalize across modalities. We introduce Atomizer, a flexible architecture that represents remote sensing images as sets of scalars, each corresponding to a spectral band value of a pixel. Each scalar is enriched with contextual metadata (acquisition time, spatial resolution, wavelength, and bandwidth), producing an atomic representation that allows a single encoder to process arbitrary modalities without interpolation or resampling. Atomizer uses structured tokenization with Fourier features and non-uniform radial basis functions to encode content and context, and maps tokens into a latent space via cross-attention. Under modality-disjoint evaluations, Atomizer outperforms standard models and demonstrates robust performance across varying resolutions and spatial sizes.