# FreeCHROMA: Generative Chroma Keying from Unaligned Background References

arXiv: https://arxiv.org/abs/2610.14445 · HTML: https://mlsanity.com/paper/2610.14445/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 10 Oct 2026.

## Builds on (most cited first, 25 of 25)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation](https://mlsanity.com/paper/2407.02371/) (2024)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [DoveNet: Deep Image Harmonization via Domain Verification](https://mlsanity.com/paper/1911.13239/) (2019)
- [Background Matting: The World is Your Green Screen](https://mlsanity.com/paper/2004.00626/) (2020)
- [RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots](https://mlsanity.com/paper/2603.04356/) (2026)
- [HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation](https://mlsanity.com/paper/2407.17438/) (2024)
- [Light-A-Video: Training-free Video Relighting via Progressive Light Fusion](https://mlsanity.com/paper/2502.08590/) (2025)
- [RelightVid: Temporal-Consistent Diffusion Model for Video Relighting](https://mlsanity.com/paper/2501.16330/) (2025)
- [Back on Track: Bundle Adjustment for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2504.14516/) (2025)
- [Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Models](https://mlsanity.com/paper/2508.12945/) (2025)
- [Bernini: Latent Semantic Planning for Video Diffusion](https://mlsanity.com/paper/2605.22344/) (2026)
- [PAI-Studio: Cinematic Video Background Replacement with Camera-Aware Motion](https://mlsanity.com/paper/2606.01399/) (2026)
- [IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation](https://mlsanity.com/paper/2506.03150/) (2025)
- [FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement](https://mlsanity.com/paper/2511.18346/) (2025)
- [ANYPORTAL: Zero-Shot Consistent Video Background Replacement](https://mlsanity.com/paper/2509.07472/) (2025)
- [ID-V2V: Identity-Preserving Video Restylization](https://mlsanity.com/paper/2607.22830/) (2026)

## Abstract (arXiv)

In film production, chroma keying composites an approved foreground performance into a new environment, but conventional workflows require coordinated background plates and dedicated harmonization. Recent generative methods often require pre-aligned backgrounds, focus on background-conditioned foreground relighting, or represent the environment only through an edited first frame. These restrictions limit background selection, scene composition, and camera motion while adding preparation or editing effort. We present FreeCHROMA, which adapts a pretrained video diffusion model in a parameter-efficient manner for generative video compositing from one to three unaligned background references. The model integrates the foreground into the referenced environment, preserving foreground identity, motion, and fine visual details while generating a spatiotemporally coherent background. Foreground-background harmonization occurs within the same generation process, without a separate relighting stage. Optional text instructions help disambiguate complex spatial relationships and camera behavior. To train the model, we construct pseudo-paired data from readily available real monocular videos through foreground-background separation and augmentation with existing foundation models. The original videos serve as training targets, while automated processing and quality verification enable scalable data construction. Across reference fidelity, foreground preservation, scene integration, harmonization, and overall video quality, FreeCHROMA consistently outperforms recent compositing and conditional video-generation baselines, with particularly clear gains under camera motion. FreeCHROMA relaxes the constraints of existing tools to enable end-to-end generative chroma keying from unaligned inputs, with exploratory experiments demonstrating its potential in robotics.