MetaEarth-MM: Unified Multimodal Remote Sensing Image Generation with Scene-centered Joint Modeling
Not on the map. Why?
Abstract (arXiv)
Multi-modal remote sensing images are vital for Earth observation, yet complete paired observations are often scarce in practice. Existing generative methods commonly address this problem through isolated pairwise modality translation, but their versatility and scalability remain limited as the number of modalities and generation tasks increases. Here, we develop a generative foundation model MetaEarth-MM for multi-modal remote sensing imagery, enabling paired joint generation and any-to-any translation across five modalities within a unified model. Recognizing the intrinsic scene consistency underlying multi-modal observations, we introduce a scene-centered joint modeling paradigm in MetaEarth-MM. Unlike previous methods that rely on direct appearance-level cross-modal mapping, our model organizes the generation around the underlying scene content. Specifically, MetaEarth-MM adopts a decoupled architecture that first infers a latent scene representation from available observations, and then generates target modalities conditioned on this intermediate state. To support training, we further construct EarthMM, a large-scale dataset comprising 2.8 million multi-resolution global images with 2.2 million aligned pairs. Extensive experiments demonstrate that MetaEarth-MM not only exhibits strong generative capability and robust generalization across diverse generation tasks, but also supports downstream tasks at both data and representation levels, highlighting its potential as a general foundation model for cross-modal Earth observation. The code and dataset will be available at https://github.com/YZPioneer/MetaEarth-MM.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 66 of its 97 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
66 of 97 references.
References · 66
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
38 more
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2025DINOv3cited by 1,414
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
- 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
- 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
- 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
- 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 639
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2023LightGlue: Local Feature Matching at Light Speedcited by 590
- 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 359
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
- 2017Functional Map of the Worldcited by 338
- 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 304
- 2022BBDM: Image-to-image Translation with Brownian Bridge Diffusion Modelscited by 225
- 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 200
- 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 187
- 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 179
- 2022DKM: Dense Kernelized Feature Matching for Geometry Estimationcited by 177
- 2023DiffusionSat: A Generative Foundation Model for Satellite Imagerycited by 131
- 2018Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Modelscited by 119
- 2019SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusioncited by 95
- 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
- 2024Foundation Models for Remote Sensing and Earth Observation: A Surveycited by 70
- 2024Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysiscited by 63
- 2024CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Modelcited by 55
- 2025Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Modelcited by 48
- 2020SpaceNet 6: Multi-Sensor All Weather Mapping Datasetcited by 47
- 2024OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flowscited by 46
- 2018The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusioncited by 42
- 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 42
- 2024Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Modelcited by 37
- 2024RSCaMa: Remote Sensing Image Change Captioning with State Space Modelcited by 35
- 2024GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesiscited by 32
- 2023Generate Your Own Scotland: Satellite Image Generation Conditioned on Mapscited by 30
- 2023RSDiff: Remote Sensing Image Generation from Text Using Diffusion Modelcited by 29
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.