# Beyond Learned Metadata-based Raw Image Reconstruction arXiv: https://arxiv.org/abs/2306.12058 ยท HTML: https://mlsanity.com/paper/2306.12058/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 56 references that are in the map. - First posted on arXiv 21 Jun 2023. ## Builds on (most cited first, 29 of 29) - [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016) - [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018) - [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016) - [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018) - [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020) - [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020) - [Lossy Image Compression with Compressive Autoencoders](https://mlsanity.com/paper/1703.00395/) (2017) - [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020) - [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021) - [Low-Light Image Enhancement with Normalizing Flow](https://mlsanity.com/paper/2109.05923/) (2021) - [Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations](https://mlsanity.com/paper/1704.00648/) (2017) - [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018) - [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020) - [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020) - [Variable Rate Deep Image Compression With a Conditional Autoencoder](https://mlsanity.com/paper/1909.04802/) (2019) - [Noise Flow: Noise Modeling with Conditional Normalizing Flows](https://mlsanity.com/paper/1908.08453/) (2019) - [Invertible Image Signal Processing](https://mlsanity.com/paper/2103.15061/) (2021) - [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018) - [Learning Convolutional Networks for Content-weighted Image Compression](https://mlsanity.com/paper/1703.10553/) (2017) - [Superpixel Segmentation with Fully Convolutional Networks](https://mlsanity.com/paper/2003.12929/) (2020) - [Rethinking Noise Synthesis and Modeling in Raw Denoising](https://mlsanity.com/paper/2110.04756/) (2021) - [Learning End-to-End Lossy Image Compression: A Benchmark](https://mlsanity.com/paper/2002.03711/) (2020) - [Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform](https://mlsanity.com/paper/2108.09551/) (2021) - [Deep Learning for HDR Imaging: State-of-the-Art and Future Trends](https://mlsanity.com/paper/2110.10394/) (2021) - [Towards Low Light Enhancement with RAW Images](https://mlsanity.com/paper/2112.14022/) (2021) - [Lossy Image Compression with Normalizing Flows](https://mlsanity.com/paper/2008.10486/) (2020) - [Learning sRGB-to-Raw-RGB De-rendering with Content-Aware Metadata](https://mlsanity.com/paper/2206.01813/) (2022) - [Raw Image Reconstruction with Learned Compact Metadata](https://mlsanity.com/paper/2302.12995/) (2023) - [Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images](https://mlsanity.com/paper/1311.6887/) (2013) ## Built on by (most cited first, 9 of 9) - [ExposureDiffusion: Learning to Expose for Low-light Image Enhancement](https://mlsanity.com/paper/2307.07710/) (2023) - [A Learnable Color Correction Matrix for RAW Reconstruction](https://mlsanity.com/paper/2409.02497/) (2024) - [RAWMamba: Unified sRGB-to-RAW De-rendering With State Space Model](https://mlsanity.com/paper/2411.11717/) (2024) - [RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling](https://mlsanity.com/paper/2605.05941/) (2026) - [Accelerating Learned Image Compression Through Modeling Neural Training Dynamics](https://mlsanity.com/paper/2505.18107/) (2025) - [Raw-JPEG Adapter: Efficient Raw Image Compression with JPEG](https://mlsanity.com/paper/2509.19624/) (2025) - [Edit-aware RAW Reconstruction](https://mlsanity.com/paper/2512.05859/) (2025) - [RAWIC: Bit-Depth Adaptive Lossless Raw Image Compression](https://mlsanity.com/paper/2603.28105/) (2026) - [MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction](https://mlsanity.com/paper/2606.24479/) (2026) ## Abstract (arXiv) While raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels, they are not widely adopted by general users due to their substantial storage requirements. Very recent studies propose to compress raw images by designing sampling masks within the pixel space of the raw image. However, these approaches often leave space for pursuing more effective image representations and compact metadata. In this work, we propose a novel framework that learns a compact representation in the latent space, serving as metadata, in an end-to-end manner. Compared with lossy image compression, we analyze the intrinsic difference of the raw image reconstruction task caused by rich information from the sRGB image. Based on the analysis, a novel backbone design with asymmetric and hybrid spatial feature resolutions is proposed, which significantly improves the rate-distortion performance. Besides, we propose a novel design of the context model, which can better predict the order masks of encoding/decoding based on both the sRGB image and the masks of already processed features. Benefited from the better modeling of the correlation between order masks, the already processed information can be better utilized. Moreover, a novel sRGB-guided adaptive quantization precision strategy, which dynamically assigns varying levels of quantization precision to different regions, further enhances the representation ability of the model. Finally, based on the iterative properties of the proposed context model, we propose a novel strategy to achieve variable bit rates using a single model. This strategy allows for the continuous convergence of a wide range of bit rates. Extensive experimental results demonstrate that the proposed method can achieve better reconstruction quality with a smaller metadata size.