# Color-aware Deep Temporal Backdrop Duplex Matting System

arXiv: https://arxiv.org/abs/2306.02954 · HTML: https://mlsanity.com/paper/2306.02954/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 61 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Deep Image Matting](https://mlsanity.com/paper/1703.03872/) (2017)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://mlsanity.com/paper/2011.11961/) (2020)
- [Background Matting: The World is Your Green Screen](https://mlsanity.com/paper/2004.00626/) (2020)
- [Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation](https://mlsanity.com/paper/1909.09725/) (2019)
- [Privacy-Preserving Portrait Matting](https://mlsanity.com/paper/2104.14222/) (2021)
- [Semantic Image Matting](https://mlsanity.com/paper/2104.08201/) (2021)
- [Deep Video Matting via Spatio-Temporal Alignment and Aggregation](https://mlsanity.com/paper/2104.11208/) (2021)
- [High-Resolution Deep Image Matting](https://mlsanity.com/paper/2009.06613/) (2020)
- [Attention-guided Temporally Coherent Video Object Matting](https://mlsanity.com/paper/2105.11427/) (2021)
- [Automatic Trimap Generation for Image Matting](https://mlsanity.com/paper/1707.00333/) (2017)
- [Hierarchical Opacity Propagation for Image Matting](https://mlsanity.com/paper/2004.03249/) (2020)
- [Fully-Trainable Deep Matching](https://mlsanity.com/paper/1609.03532/) (2016)
- [ROAM: a Rich Object Appearance Model with Application to Rotoscoping](https://mlsanity.com/paper/1612.01495/) (2016)
- [DeepMatching: Hierarchical Deformable Dense Matching](https://mlsanity.com/paper/1506.07656/) (2015)

## Built on by (most cited first, 1 of 1)

- [Capture Stage Matting: Challenges, Approaches, and Solutions for Offline and Real-Time Processing](https://mlsanity.com/paper/2507.07623/) (2025)

## Abstract (arXiv)

Deep learning-based alpha matting showed tremendous improvements in recent years, yet, feature film production studios still rely on classical chroma keying including costly post-production steps. This perceived discrepancy can be explained by some missing links necessary for production which are currently not adequately addressed in the alpha matting community, in particular foreground color estimation or color spill compensation. We propose a neural network-based temporal multi-backdrop production system that combines beneficial features from chroma keying and alpha matting. Given two consecutive frames with different background colors, our one-encoder-dual-decoder network predicts foreground colors and alpha values using a patch-based overlap-blend approach. The system is able to handle imprecise backdrops, dynamic cameras, and dynamic foregrounds and has no restrictions on foreground colors. We compare our method to state-of-the-art algorithms using benchmark datasets and a video sequence captured by a demonstrator setup. We verify that a dual backdrop input is superior to the usually applied trimap-based approach. In addition, the proposed studio set is actor friendly, and produces high-quality, temporal consistent alpha and color estimations that include a superior color spill compensation.