# Independent Samples, Correlated Variance A Learnable Cross-View Cue in Path-Traced Stereo Data

arXiv: https://arxiv.org/abs/2606.25483 · HTML: https://mlsanity.com/paper/2606.25483/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 10 of 10)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [What Makes Good Synthetic Training Data for Zero-Shot Stereo Matching?](https://mlsanity.com/paper/2504.16930/) (2025)
- [Achieving Domain Robustness in Stereo Matching Networks by Removing Shortcut Learning](https://mlsanity.com/paper/2106.08486/) (2021)

## Abstract (arXiv)

Path-traced synthetic stereo is a primary training substrate for disparity networks, and the pipelines that consume it assume Monte~Carlo (MC) rendering noise is independent across the two views. The assumption is correct at the level it is stated---individual samples---but silent about the object a network actually sees. We show that the per-pixel MC variance fields, though built from independent samples, are strongly correlated once aligned by the ground-truth disparity, and that a network can learn to match with that correlation. Across 20 indoor scenes the warped correlation is 0.754±0.016 against 0.360 unwarped; it replicates on a second renderer with a different sampler and sample budget (0.743±0.044), and a seed-count analysis puts the population value near 0.85, making the measurement a lower bound. The effect is field-level, not sample-level: at the warp correspondence the signed per-seed residual correlation is -0.0004 while the residual envelope correlates at 0.38. A capacity-limited siamese probe given nothing but variance-field patches reaches 78.8% two-alternative forced-choice accuracy on held-out scenes and falls to chance once the alignment is destroyed; under the ordinary single-render condition, decorrelation still costs 2.40 percentage points in a difference-in-differences design, positive in all six held-out scenes. A real sensor's variance is fixed by its own signal rather than by transport difficulty, so across views it is redundant with intensity: the cue is specific to rendered data.