L1 Estimation: On the Optimality of Linear Estimators

Consider the problem of estimating a random variable X from noisy observations Y = X+ Z, where Z is standard normal, under the L1 fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on X that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution PX|Y=y is symmetric for all y, then X must follow a Gaussian distribution. Additionally, we consider other Lp losses and observe the following phenomenon: for p ∈ [1,2], Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for p ∈ (2,∞), infinitely many prior distributions on X can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.