# Self-Supervised Continuous Colormap Recovery from a 2D Scalar Field Visualization without a Legend

arXiv: https://arxiv.org/abs/2507.20632 · HTML: https://mlsanity.com/paper/2507.20632/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 68 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [VIS30K: A Collection of Figures and Tables from IEEE Visualization Conference Publications](https://mlsanity.com/paper/2101.01036/) (2020)
- [VisCode: Embedding Information in Visualization Images using Encoder-Decoder Network](https://mlsanity.com/paper/2009.03817/) (2020)
- [Deep Colormap Extraction from Visualizations](https://mlsanity.com/paper/2103.00741/) (2021)
- [ParamsDrag: Interactive Parameter Space Exploration via Image-Space Dragging](https://mlsanity.com/paper/2407.14100/) (2024)
- [Optimizing colormaps with consideration for color vision deficiency to enable accurate interpretation of scientific data](https://mlsanity.com/paper/1712.01662/) (2017)
- [Void-and-Cluster Sampling of Large Scattered Data and Trajectories](https://mlsanity.com/paper/1907.05073/) (2019)
- [Scalable Topological Data Analysis and Visualization for Evaluating Data-Driven Models in Scientific Applications](https://mlsanity.com/paper/1907.08325/) (2019)
- [Color Crafting: Automating the Construction of Designer Quality Color Ramps](https://mlsanity.com/paper/1908.00629/) (2019)
- [Palettailor: Discriminable Colorization for Categorical Data](https://mlsanity.com/paper/2009.02969/) (2020)
- [A Testing Environment for Continuous Colormaps](https://mlsanity.com/paper/2009.13133/) (2020)
- [Color Orchestra: Ordering Color Palettes for Interpolation and Prediction](https://mlsanity.com/paper/1703.06003/) (2017)
- [Spectral Visualization Sharpening](https://mlsanity.com/paper/1907.10208/) (2019)
- [Science in a Blink: Supporting Ensemble Perception in Scalar Fields](https://mlsanity.com/paper/2406.14452/) (2024)

## Built on by (most cited first, 1 of 1)

- [Beyond the Good, the Bad, and the Ugly: Colormap Assessment through Data-Aware Perceptual Metric](https://mlsanity.com/paper/2610.03720/) (2026)

## Abstract (arXiv)

Recovering a continuous colormap from a single 2D scalar field visualization can be quite challenging, especially in the absence of a corresponding color legend. In this paper, we propose a novel colormap recovery approach that extracts the colormap from a color-encoded 2D scalar field visualization by simultaneously predicting the colormap and underlying data using a decoupling-and-reconstruction strategy. Our approach first separates the input visualization into colormap and data using a decoupling module, then reconstructs the visualization with a differentiable color-mapping module. To guide this process, we design a reconstruction loss between the input and reconstructed visualizations, which serves both as a constraint to ensure strong correlation between colormap and data during training, and as a self-supervised optimizer for fine-tuning the predicted colormap of unseen visualizations during inferencing. To ensure smoothness and correct color ordering in the extracted colormap, we introduce a compact colormap representation using cubic B-spline curves and an associated color order loss. We evaluate our method quantitatively and qualitatively on a synthetic dataset and a collection of real-world visualizations from the VIS30K dataset. Additionally, we demonstrate its utility in two prototype applications -- colormap adjustment and colormap transfer -- and explore its generalization to visualizations with color legends and ones encoded using discrete color palettes.