# Blind Augmentation: Calibration-free Camera Distortion Model Estimation for Real-time Mixed-reality Consistency

arXiv: https://arxiv.org/abs/2503.01387 · HTML: https://mlsanity.com/paper/2503.01387/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 40 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Image Harmonization](https://mlsanity.com/paper/1703.00069/) (2017)
- [A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift](https://mlsanity.com/paper/2206.10810/) (2022)
- [Learning to Deblur using Light Field Generated and Real Defocus Images](https://mlsanity.com/paper/2204.00367/) (2022)
- [Noise2NoiseFlow: Realistic Camera Noise Modeling without Clean Images](https://mlsanity.com/paper/2206.01103/) (2022)

## Built on by (most cited first, 1 of 1)

- [Parameter-Free Neural Lens Blur Rendering for High-Fidelity Composites](https://mlsanity.com/paper/2511.17014/) (2025)

## Abstract (arXiv)

Real camera footage is subject to noise, motion blur (MB) and depth of field (DoF). In some applications these might be considered distortions to be removed, but in others it is important to model them because it would be ineffective, or interfere with an aesthetic choice, to simply remove them. In augmented reality applications where virtual content is composed into a live video feed, we can model noise, MB and DoF to make the virtual content visually consistent with the video. Existing methods for this typically suffer two main limitations. First, they require a camera calibration step to relate a known calibration target to the specific cameras response. Second, existing work require methods that can be (differentiably) tuned to the calibration, such as slow and specialized neural networks. We propose a method which estimates parameters for noise, MB and DoF instantly, which allows using off-the-shelf real-time simulation methods from e.g., a game engine in compositing augmented content. Our main idea is to unlock both features by showing how to use modern computer vision methods that can remove noise, MB and DoF from the video stream, essentially providing self-calibration. This allows to auto-tune any black-box real-time noise+MB+DoF method to deliver fast and high-fidelity augmentation consistency.