# Consistency Analysis of an Empirical Minimum Error Entropy Algorithm

arXiv: https://arxiv.org/abs/1412.5272 · HTML: https://mlsanity.com/paper/1412.5272/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2014.

## Built on by (most cited first, 7 of 7)

- [Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions](https://mlsanity.com/paper/2107.00896/) (2021)
- [Robust Kernel-based Distribution Regression](https://mlsanity.com/paper/2104.10637/) (2021)
- [Maximum Correntropy Criterion Regression models with tending-to-zero scale parameters](https://mlsanity.com/paper/2110.12751/) (2021)
- [Stability-based Generalization Analysis of Randomized Coordinate Descent for Pairwise Learning](https://mlsanity.com/paper/2503.01530/) (2025)
- [Understanding Robust Machine Learning for Nonparametric Regression with Heavy-Tailed Noise](https://mlsanity.com/paper/2510.09888/) (2025)
- [Deep regression learning from dependent observations with minimum error entropy principle](https://mlsanity.com/paper/2603.11138/) (2026)
- [Meta Additive Model: Interpretable Sparse Learning With Auto Weighting](https://mlsanity.com/paper/2604.20111/) (2026)

## Abstract (arXiv)

In this paper we study the consistency of an empirical minimum error entropy (MEE) algorithm in a regression setting. We introduce two types of consistency. The error entropy consistency, which requires the error entropy of the learned function to approximate the minimum error entropy, is shown to be always true if the bandwidth parameter tends to 0 at an appropriate rate. The regression consistency, which requires the learned function to approximate the regression function, however, is a complicated issue. We prove that the error entropy consistency implies the regression consistency for homoskedastic models where the noise is independent of the input variable. But for heteroskedastic models, a counterexample is used to show that the two types of consistency do not coincide. A surprising result is that the regression consistency is always true, provided that the bandwidth parameter tends to infinity at an appropriate rate. Regression consistency of two classes of special models is shown to hold with fixed bandwidth parameter, which further illustrates the complexity of regression consistency of MEE. Fourier transform plays crucial roles in our analysis.