# Keeping Score: Adaptive, Tuning-Free Loss Weighting for Score-Augmented Neural Ratio Estimation

arXiv: https://arxiv.org/abs/2605.12118 · HTML: https://mlsanity.com/paper/2605.12118/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 20 of 20)

- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Flexible statistical inference for mechanistic models of neural dynamics](https://mlsanity.com/paper/1711.01861/) (2017)
- [On Contrastive Learning for Likelihood-free Inference](https://mlsanity.com/paper/2002.03712/) (2020)
- [Approximating Likelihood Ratios with Calibrated Discriminative Classifiers](https://mlsanity.com/paper/1506.02169/) (2015)
- [Mining gold from implicit models to improve likelihood-free inference](https://mlsanity.com/paper/1805.12244/) (2018)
- [Student-t Processes as Alternatives to Gaussian Processes](https://mlsanity.com/paper/1402.4306/) (2014)
- [Neural Methods for Amortized Inference](https://mlsanity.com/paper/2404.12484/) (2024)
- [Likelihood-Free Parameter Estimation with Neural Bayes Estimators](https://mlsanity.com/paper/2208.12942/) (2022)
- [Truncated Marginal Neural Ratio Estimation](https://mlsanity.com/paper/2107.01214/) (2021)
- [Neural Networks for Parameter Estimation in Intractable Models](https://mlsanity.com/paper/2107.14346/) (2021)
- [Towards Reliable Simulation-Based Inference with Balanced Neural Ratio Estimation](https://mlsanity.com/paper/2208.13624/) (2022)
- [Fast covariance parameter estimation of spatial Gaussian process models using neural networks](https://mlsanity.com/paper/2012.15339/) (2020)
- [Neural Bayes Estimators for Irregular Spatial Data using Graph Neural Networks](https://mlsanity.com/paper/2310.02600/) (2023)
- [Neural Likelihood Surfaces for Spatial Processes with Computationally Intensive or Intractable Likelihoods](https://mlsanity.com/paper/2305.04634/) (2023)
- [Neural Posterior Estimation with Differentiable Simulators](https://mlsanity.com/paper/2207.05636/) (2022)
- [Neural Bayes estimators for censored inference with peaks-over-threshold models](https://mlsanity.com/paper/2306.15642/) (2023)
- [Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks](https://mlsanity.com/paper/2311.11913/) (2023)
- [Neural Parameter Estimation with Incomplete Data](https://mlsanity.com/paper/2501.04330/) (2025)

## Abstract (arXiv)

Neural likelihood surrogates (e.g., Neural Ratio Estimation) for stochastic process models are commonly trained via probabilistic classification on simulated data, which forces a tradeoff between surrogate quality and training costs. For structured models where the exact score nablaθ log p(x mid θ) is available, this information can be incorporated into training by augmenting the cross-entropy loss with a score-matching term. However, the optimal weighting of the two losses is not known a priori, and selecting it by hand requires expensive tuning that undercuts the computational savings. We propose an adaptive, tuning-free algorithm that sets the score loss weights during training based on loss gradients, adding minimal overhead to standard classifier training. We evaluate our approach on case studies involving network dynamics and spatial processes, demonstrating that it improves surrogate quality at a drastically lower computational cost than generating more training data. Notably, in some cases, our approach achieves downstream inference performance equivalent to a 10x increase in training data with less than a 1.1x increase in training time.