mlsanityData to 14 Oct 2026

Small area learning

arXiv 2610.15953 ↗First posted 11 Oct 2026stat.ME cs.LG math.STcited by 0

On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation

Abstract (arXiv)

Small area estimation (SAE) traditionally borrows strength across domains through a prespecified working model. We introduce small area learning (SAL), a learner-agnostic framework that formulates SAE as a multi-target semi-supervised learning problem and treats outcome prediction as a nuisance task. Areas may differ through unrestricted covariate shift and structured conditional shift, with information shared through common and area-specific components. We develop standard and cross-fitted estimators for low-complexity and flexible learners, unifying synthetic, model-based, and fully residual-corrected estimation. For area means, we establish consistency and asymptotic normality and provide variance estimators and confidence intervals. We extend the theory to settings with only aggregated population covariates and characterize the resulting aggregation bias. SAL reveals a bias--variance trade-off: partial residual correction can reduce mean squared error, whereas full correction yields estimators robust to working-model misspecification, potentially at higher variance. Simulations yield RMSE reductions of 34%--49% relative to direct estimation, with the best configuration varying across settings. An empirical application favours a cross-fitted estimator using a flexible learner under the chosen validation criterion. The numerical studies also illustrate how shrinkage can produce overly optimistic confidence intervals at the expense of strong modeling assumptions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. First posted on arXiv 11 Oct 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fully Bayesian Spectral Clustering and Benchmarking with Uncertainty Quantification for Small Area Estimation
  2. 2025Incorporating priors in learning: a random matrix study under a teacher-student framework
  3. 2025Mind the (Data) Gap: Evaluating Vision Systems in Small Data Applications
  4. 2026Interactions Between Crosscoder Features: A Compact Proofs Perspective
  5. 2026Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects
  6. 2025SAEs Are Good for Steering -- If You Select the Right Features
  7. 2026Information Theory and Statistical Learning
  8. 2026Rescuing Points Trapped Inside Incorrect Decision Regions: A target for Evaluating and Improving Forward-Forward Learning