# An End-to-End Real-World Camera Imaging Pipeline

arXiv: https://arxiv.org/abs/2411.10773 · HTML: https://mlsanity.com/paper/2411.10773/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 16 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Checkerboard Context Model for Efficient Learned Image Compression](https://mlsanity.com/paper/2103.15306/) (2021)
- [Is Local SGD Better than Minibatch SGD?](https://mlsanity.com/paper/2002.07839/) (2020)
- [Conditional Probability Models for Deep Image Compression](https://mlsanity.com/paper/1801.04260/) (2018)
- [MLIC: Multi-Reference Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2211.07273/) (2022)
- [Multi-modal Queried Object Detection in the Wild](https://mlsanity.com/paper/2305.18980/) (2023)
- [AIM 2020 Challenge on Learned Image Signal Processing Pipeline](https://mlsanity.com/paper/2011.04994/) (2020)
- [Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision](https://mlsanity.com/paper/2108.08119/) (2021)
- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [CameraNet: A Two-Stage Framework for Effective Camera ISP Learning](https://mlsanity.com/paper/1908.01481/) (2019)
- [AWNet: Attentive Wavelet Network for Image ISP](https://mlsanity.com/paper/2008.09228/) (2020)
- [Distilling Semantic Priors from SAM to Efficient Image Restoration Models](https://mlsanity.com/paper/2403.16368/) (2024)
- [Learnability Enhancement for Low-light Raw Denoising: Where Paired Real Data Meets Noise Modeling](https://mlsanity.com/paper/2207.06103/) (2022)
- [Monte Carlo DropBlock for Modelling Uncertainty in Object Detection](https://mlsanity.com/paper/2108.03614/) (2021)
- [Deep Optimal Transport: A Practical Algorithm for Photo-realistic Image Restoration](https://mlsanity.com/paper/2306.02342/) (2023)
- [Computational Politeness in Natural Language Processing: A Survey](https://mlsanity.com/paper/2407.12814/) (2024)
- [Learning an Inverse Tone Mapping Network with a Generative Adversarial Regularizer](https://mlsanity.com/paper/1804.07677/) (2018)
- [RAWtoBit: A Fully End-to-end Camera ISP Network](https://mlsanity.com/paper/2208.07639/) (2022)

## Built on by (most cited first, 3 of 3)

- [Smoothness Really Matters: A Simple Yet Effective Approach for Unsupervised Graph Domain Adaptation](https://mlsanity.com/paper/2412.11654/) (2024)
- [Intra and Inter Parser-Prompted Transformers for Effective Image Restoration](https://mlsanity.com/paper/2503.14037/) (2025)
- [Learning Differential Pyramid Representation for Tone Mapping](https://mlsanity.com/paper/2412.01463/) (2024)

## Abstract (arXiv)

Recent advances in neural camera imaging pipelines have demonstrated notable progress. Nevertheless, the real-world imaging pipeline still faces challenges including the lack of joint optimization in system components, computational redundancies, and optical distortions such as lens shading.In light of this, we propose an end-to-end camera imaging pipeline (RealCamNet) to enhance real-world camera imaging performance. Our methodology diverges from conventional, fragmented multi-stage image signal processing towards end-to-end architecture. This architecture facilitates joint optimization across the full pipeline and the restoration of coordinate-biased distortions. RealCamNet is designed for high-quality conversion from RAW to RGB and compact image compression. Specifically, we deeply analyze coordinate-dependent optical distortions, e.g., vignetting and dark shading, and design a novel Coordinate-Aware Distortion Restoration (CADR) module to restore coordinate-biased distortions. Furthermore, we propose a Coordinate-Independent Mapping Compression (CIMC) module to implement tone mapping and redundant information compression. Existing datasets suffer from misalignment and overly idealized conditions, making them inadequate for training real-world imaging pipelines. Therefore, we collected a real-world imaging dataset. Experiment results show that RealCamNet achieves the best rate-distortion performance with lower inference latency.