# Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networks

arXiv: https://arxiv.org/abs/2208.04441 · HTML: https://mlsanity.com/paper/2208.04441/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 87 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks](https://mlsanity.com/paper/1711.10485/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [Exploring Models and Data for Remote Sensing Image Caption Generation](https://mlsanity.com/paper/1712.07835/) (2017)
- [DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis](https://mlsanity.com/paper/2008.05865/) (2020)
- [LAFITE: Towards Language-Free Training for Text-to-Image Generation](https://mlsanity.com/paper/2111.13792/) (2021)
- [Robust Classification with Convolutional Prototype Learning](https://mlsanity.com/paper/1805.03438/) (2018)
- [Learning What and Where to Draw](https://mlsanity.com/paper/1610.02454/) (2016)
- [CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP](https://mlsanity.com/paper/2110.11316/) (2021)

## Built on by (most cited first, 30 of 30)

- [RSGPT: A Remote Sensing Vision Language Model and Benchmark](https://mlsanity.com/paper/2307.15266/) (2023)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model](https://mlsanity.com/paper/2403.11614/) (2024)
- [Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model](https://mlsanity.com/paper/2501.00895/) (2025)
- [Conformal Prediction for Time Series with Modern Hopfield Networks](https://mlsanity.com/paper/2303.12783/) (2023)
- [Changes to Captions: An Attentive Network for Remote Sensing Change Captioning](https://mlsanity.com/paper/2304.01091/) (2023)
- [SpectralMamba: Efficient Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2404.08489/) (2024)
- [GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis](https://mlsanity.com/paper/2404.06637/) (2024)
- [RSDiff: Remote Sensing Image Generation from Text Using Diffusion Model](https://mlsanity.com/paper/2309.02455/) (2023)
- [Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives](https://mlsanity.com/paper/2404.08926/) (2024)
- [Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2410.08613/) (2024)
- [AI Security for Geoscience and Remote Sensing: Challenges and Future Trends](https://mlsanity.com/paper/2212.09360/) (2022)
- [RSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering](https://mlsanity.com/paper/2310.13120/) (2023)
- [EarthSynth: Generating Informative Earth Observation with Diffusion Models](https://mlsanity.com/paper/2505.12108/) (2025)
- [Long-Term Visual Object Tracking with Event Cameras: An Associative Memory Augmented Tracker and A Benchmark Dataset](https://mlsanity.com/paper/2403.05839/) (2024)
- [CC-Diff: Enhancing Contextual Coherence in Remote Sensing Image Synthesis](https://mlsanity.com/paper/2412.08464/) (2024)
- [ChangeMinds: Multi-task Framework for Detecting and Describing Changes in Remote Sensing](https://mlsanity.com/paper/2410.10047/) (2024)
- [ERDDCI: Exact Reversible Diffusion via Dual-Chain Inversion for High-Quality Image Editing](https://mlsanity.com/paper/2410.14247/) (2024)
- [FICGen: Frequency-Inspired Contextual Disentanglement for Layout-driven Degraded Image Generation](https://mlsanity.com/paper/2509.01107/) (2025)
- [Dsfer-Net: A Deep Supervision and Feature Retrieval Network for Bitemporal Change Detection Using Modern Hopfield Networks](https://mlsanity.com/paper/2304.01101/) (2023)
- [Generating Synthetic Satellite Imagery for Rare Objects: An Empirical Comparison of Models and Metrics](https://mlsanity.com/paper/2409.01138/) (2024)
- [TerraDiT-Omega: Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive](https://mlsanity.com/paper/2606.31029/) (2026)
- [Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques](https://mlsanity.com/paper/2410.17283/) (2024)
- [Towards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion](https://mlsanity.com/paper/2601.15829/) (2026)
- [Uni-RS: A Spatially Faithful Unified Understanding and Generation Model for Remote Sensing](https://mlsanity.com/paper/2601.17673/) (2026)
- [TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesis](https://mlsanity.com/paper/2603.02172/) (2026)
- [Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition](https://mlsanity.com/paper/2607.06185/) (2026)
- [GeoCore-9B: Towards Geo-Aware Generative Foundation Models in Earth Observation](https://mlsanity.com/paper/2608.01896/) (2026)
- [OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation](https://mlsanity.com/paper/2610.15414/) (2026)
- [Omni-RS: Unifying Multi-Task Understanding and Generation in Remote Sensing](https://mlsanity.com/paper/2610.18323/) (2026)

## Abstract (arXiv)

The synthesis of high-resolution remote sensing images based on text descriptions has great potential in many practical application scenarios. Although deep neural networks have achieved great success in many important remote sensing tasks, generating realistic remote sensing images from text descriptions is still very difficult. To address this challenge, we propose a novel text-to-image modern Hopfield network (Txt2Img-MHN). The main idea of Txt2Img-MHN is to conduct hierarchical prototype learning on both text and image embeddings with modern Hopfield layers. Instead of directly learning concrete but highly diverse text-image joint feature representations for different semantics, Txt2Img-MHN aims to learn the most representative prototypes from text-image embeddings, achieving a coarse-to-fine learning strategy. These learned prototypes can then be utilized to represent more complex semantics in the text-to-image generation task. To better evaluate the realism and semantic consistency of the generated images, we further conduct zero-shot classification on real remote sensing data using the classification model trained on synthesized images. Despite its simplicity, we find that the overall accuracy in the zero-shot classification may serve as a good metric to evaluate the ability to generate an image from text. Extensive experiments on the benchmark remote sensing text-image dataset demonstrate that the proposed Txt2Img-MHN can generate more realistic remote sensing images than existing methods. Code and pre-trained models are available online (https://github.com/YonghaoXu/Txt2Img-MHN).