# Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems

arXiv: https://arxiv.org/abs/2605.22891 · HTML: https://mlsanity.com/paper/2605.22891/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 21 May 2026.

## Builds on (most cited first, 28 of 28)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [Real-time gravitational-wave science with neural posterior estimation](https://mlsanity.com/paper/2106.12594/) (2021)
- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [normflows: A PyTorch Package for Normalizing Flows](https://mlsanity.com/paper/2302.12014/) (2023)
- [Probabilistic Conformal Prediction Using Conditional Random Samples](https://mlsanity.com/paper/2206.06584/) (2022)
- [Sampling-Based Accuracy Testing of Posterior Estimators for General Inference](https://mlsanity.com/paper/2302.03026/) (2023)
- [Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics](https://mlsanity.com/paper/2405.14806/) (2024)
- [Calibrated Reliable Regression using Maximum Mean Discrepancy](https://mlsanity.com/paper/2006.10255/) (2020)
- [A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful](https://mlsanity.com/paper/2110.06581/) (2021)
- [A Lorentz-Equivariant Transformer for All of the LHC](https://mlsanity.com/paper/2411.00446/) (2024)
- [The Landscape of Unfolding with Machine Learning](https://mlsanity.com/paper/2404.18807/) (2024)
- [SPANet: Generalized Permutationless Set Assignment for Particle Physics using Symmetry Preserving Attention](https://mlsanity.com/paper/2106.03898/) (2021)
- [Statistically-informed deep learning for gravitational wave parameter estimation](https://mlsanity.com/paper/1903.01998/) (2019)
- [nu2-Flows: Fast and improved neutrino reconstruction in multi-neutrino final states with conditional normalizing flows](https://mlsanity.com/paper/2307.02405/) (2023)
- [Topological Reconstruction of Particle Physics Processes using Graph Neural Networks](https://mlsanity.com/paper/2303.13937/) (2023)
- [Another Fit Bites the Dust: Conformal Prediction as a Calibration Standard for Machine Learning in High-Energy Physics](https://mlsanity.com/paper/2512.17048/) (2025)

## Built on by (most cited first, 1 of 1)

- [No Free Lunch in Flow Surrogates under Time-Varying Boundary Conditions: A Two-Regime Study](https://mlsanity.com/paper/2607.23667/) (2026)

## Abstract (arXiv)

Evaluation in scientific reconstruction is dominated by pointwise metrics - RMSE, MAE, per-event resolution - under the implicit assumption that lower error means better reconstruction. We show that this assumption fails structurally for inverse problems with multimodal posteriors. By the law of total variance, point estimators trained to minimize MSE or MAE produce a marginal spectrum strictly narrower than the truth whenever the posterior has nonzero width. The resulting bias is independent of architecture, training, and dataset size, and it compresses precisely the spectral features - tails, modes, shapes - that downstream scientific measurements rely on. We propose a three-part evaluation protocol where each step targets a failure mode the others miss: per-event distributional accuracy via CRPS, population-level marginal accuracy via a spectrum-fidelity diagnostic, and uncertainty trustworthiness via coverage-based calibration. On a synthetic benchmark with an analytic posterior and on a realistic many-to-one inverse problem from particle physics, model rankings reverse between pointwise and distributional metrics, and calibration further separates architectures indistinguishable under CRPS. The evaluation protocol, not the model, determines the scientific conclusion.