The Arbitrary-Placement Problem in Entropy-Minimizing Selection, and a Residual-Entropy Formulation

Entropy-based selection objectives suffer from a fundamental degeneracy: minimizing Shannon entropy H(pA) rewards confident selection regardless of whether the selected candidate is informative. We address this limitation with the residual entropy D = H(pA) - H(pβ), where pβ is induced by candidate trust weights. We prove the exact identity D = -KL(pAVert pβ) - Delta, where Delta measures whether the score-induced distribution and trust profile favor the same candidates. Boundary cases establish basic safety: under uniform trust, D≤0 automatically, so an equal-trust, non-starving state is never penalized, while at any one-hot limit, D→0 regardless of the selected candidate. For the intermediate regime where selection occurs, we prove that D≤0 when candidate ordering by trust agrees pairwise with ordering by informativeness, and derive a tighter certificate based on the leading candidate's margin over its competitors. These results are independent of the candidate-scoring function and apply to both stationary and dynamically changing information. Experiments with a gradient-based mixture-of-experts router confirm that the ordering conditions can hold during real optimization and show that correct ordering improves downstream performance when candidates are non-interchangeable and selections are used directly rather than averaged. Beyond routing, margin-based reweighting matches or outperforms fixed-strength baselines in a class-imbalance task, while informative selection in a production video-prediction system reduces MSE by approximately 20% and transfers to a related species. Residual entropy, therefore, provides a safety criterion for selection and a usable signal for deciding when that selection is informative.