# An Alternative Probabilistic Interpretation of the Huber Loss

arXiv: https://arxiv.org/abs/1911.02088 · HTML: https://mlsanity.com/paper/1911.02088/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2019.

## Built on by (most cited first, 30 of 43)

- [Balanced MSE for Imbalanced Visual Regression](https://mlsanity.com/paper/2203.16427/) (2022)
- [A Comprehensive Survey of Regression Based Loss Functions for Time Series Forecasting](https://mlsanity.com/paper/2211.02989/) (2022)
- [Foundation Models for Time Series: A Survey](https://mlsanity.com/paper/2504.04011/) (2025)
- [Instance-Dependent Generalization Bounds via Optimal Transport](https://mlsanity.com/paper/2211.01258/) (2022)
- [PhysioWave: A Multi-Scale Wavelet-Transformer for Physiological Signal Representation](https://mlsanity.com/paper/2506.10351/) (2025)
- [VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantification](https://mlsanity.com/paper/2405.20905/) (2024)
- [CIRA Guide to Custom Loss Functions for Neural Networks in Environmental Sciences -- Version 1](https://mlsanity.com/paper/2106.09757/) (2021)
- [QuantumLeak: Stealing Quantum Neural Networks from Cloud-based NISQ Machines](https://mlsanity.com/paper/2403.10790/) (2024)
- [Avocado Price Prediction Using a Hybrid Deep Learning Model: TCN-MLP-Attention Architecture](https://mlsanity.com/paper/2505.09907/) (2025)
- [Variational Sparse Coding with Learned Thresholding](https://mlsanity.com/paper/2205.03665/) (2022)
- [FusionViT: Hierarchical 3D Object Detection via LiDAR-Camera Vision Transformer Fusion](https://mlsanity.com/paper/2311.03620/) (2023)
- [Deep Variational Bayesian Modeling of Haze Degradation Process](https://mlsanity.com/paper/2412.03745/) (2024)
- [TimeCapsule: Solving the Jigsaw Puzzle of Long-Term Time Series Forecasting with Compressed Predictive Representations](https://mlsanity.com/paper/2504.12721/) (2025)
- [Beyond Structure: Invariant Crystal Property Prediction with Pseudo-Particle Ray Diffraction](https://mlsanity.com/paper/2509.21778/) (2025)
- [EGR: Equivariant Graph Refinement and Assessment of 3D Protein Complex Structures](https://mlsanity.com/paper/2205.10390/) (2022)
- [A novel Deep Learning approach for one-step Conformal Prediction approximation](https://mlsanity.com/paper/2207.12377/) (2022)
- [RDDPM: Robust Denoising Diffusion Probabilistic Model for Unsupervised Anomaly Segmentation](https://mlsanity.com/paper/2508.02903/) (2025)
- [Sparse Bayesian Lasso via a Variable-Coefficient ℓ1 Penalty](https://mlsanity.com/paper/2211.05089/) (2022)
- [A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning](https://mlsanity.com/paper/2401.02325/) (2024)
- [Exploiting Digital Surface Models for Inferring Super-Resolution for Remotely Sensed Images](https://mlsanity.com/paper/2205.04056/) (2022)
- [Adaptive Output Steps: FlexiSteps Network for Dynamic Trajectory Prediction](https://mlsanity.com/paper/2508.17797/) (2025)
- [MINT-RVAE: Multi-Cues Intention Prediction of Human-Robot Interaction using Human Pose and Emotion Information from RGB-only Camera Data](https://mlsanity.com/paper/2509.22573/) (2025)
- [RINS-T: Robust Implicit Neural Solvers for Time Series Linear Inverse Problems](https://mlsanity.com/paper/2510.17396/) (2025)
- [Real-Time Human-Robot Interaction Intent Detection Using RGB-based Pose and Emotion Cues with Cross-Camera Model Generalization](https://mlsanity.com/paper/2512.17958/) (2025)
- [TAIHRI: Task-Aware 3D Human Keypoints Localization for Close-Range Human-Robot Interaction](https://mlsanity.com/paper/2604.08921/) (2026)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Markov subsampling based Huber Criterion](https://mlsanity.com/paper/2112.06134/) (2021)
- [Cost-aware Duration Prediction for Software Upgrades in Datacenters](https://mlsanity.com/paper/2212.05155/) (2022)
- [Changes by Butterflies: Farsighted Forecasting with Group Reservoir Transformer](https://mlsanity.com/paper/2402.09573/) (2024)
- [Multi-Sensor Diffusion-Driven Optical Image Translation for Large-Scale Applications](https://mlsanity.com/paper/2404.11243/) (2024)

## Abstract (arXiv)

The Huber loss is a robust loss function used for a wide range of regression tasks. To utilize the Huber loss, a parameter that controls the transitions from a quadratic function to an absolute value function needs to be selected. We believe the standard probabilistic interpretation that relates the Huber loss to the Huber density fails to provide adequate intuition for identifying the transition point. As a result, a hyper-parameter search is often necessary to determine an appropriate value. In this work, we propose an alternative probabilistic interpretation of the Huber loss, which relates minimizing the loss to minimizing an upper-bound on the Kullback-Leibler divergence between Laplace distributions, where one distribution represents the noise in the ground-truth and the other represents the noise in the prediction. In addition, we show that the parameters of the Laplace distributions are directly related to the transition point of the Huber loss. We demonstrate, through a toy problem, that the optimal transition point of the Huber loss is closely related to the distribution of the noise in the ground-truth data. As a result, our interpretation provides an intuitive way to identify well-suited hyper-parameters by approximating the amount of noise in the data, which we demonstrate through a case study and experimentation on the Faster R-CNN and RetinaNet object detectors.