mlsanityData to 7 Oct 2026

An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation

arXiv 1901.00555 ↗First posted 2 Jan 2019cs.IT cs.LG math.ITcited by 30

Not on the map. Why?

Abstract (arXiv)

Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous information-theoretic tools have been proposed for this purpose, the oldest one remains arguably the most versatile and widespread: Fano's inequality. In this chapter, we provide a survey of Fano's inequality and its variants in the context of statistical estimation, adopting a versatile framework that covers a wide range of specific problems. We present a variety of key tools and techniques used for establishing impossibility results via this approach, and provide representative examples covering group testing, graphical model selection, sparse linear regression, density estimation, and convex optimization.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 3; 2024: 7; 2025: 3; 2026: 7.

30 papers.

Sort:

By area:

  1. 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
  2. 2024How Much Can RAG Help the Reasoning of LLM?cited by 31
  3. 2022Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variancecited by 22
  4. 2023Fairness through Aleatoric Uncertaintycited by 16
  5. 2021Towards Sample-Optimal Compressive Phase Retrieval with Sparse and Generative Priorscited by 14
  6. 2022Non-Iterative Recovery from Nonlinear Observations using Generative Modelscited by 9
  7. 2024PRSA: Prompt Stealing Attacks against Real-World Prompt Servicescited by 9
  8. 2021The Complexity of Sparse Tensor PCAcited by 7
  9. 2021Robust 1-bit Compressive Sensing with Partial Gaussian Circulant Matrices and Generative Priorscited by 7
  10. 2022On the Statistical Complexity of Estimation and Testing under Privacy Constraintscited by 6
  11. 2023RecFusion: A Binomial Diffusion Process for 1D Data for Recommendationcited by 6
  12. 2024How many samples are needed to train a deep neural network?cited by 5
18 more
  1. 2022On the well-spread property and its relation to linear regressioncited by 4
  2. 2024Retraining with Predicted Hard Labels Provably Increases Model Accuracycited by 4
  3. 2022On Average-Case Error Bounds for Kernel-Based Bayesian Quadraturecited by 3
  4. 2024On Bits and Bandits: Quantifying the Regret-Information Trade-offcited by 3
  5. 2024Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Datacited by 3
  6. 2025Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoningcited by 3
  7. 2023Blocked Collaborative Bandits: Online Collaborative Filtering with Per-Item Budget Constraintscited by 2
  8. 2025Learning Single Index Models with Diffusion Priorscited by 2
  9. 2026Tackling the Inherent Difficulty of Noise Filtering in RAGcited by 2
  10. 2026Adaptive Confidence Regularization for Multimodal Failure Detectioncited by 1
  11. 2022Entropy-driven Unsupervised Keypoint Representation Learning in Videoscited by 0
  12. 2024Contrastive Learning to Fine-Tune Feature Extraction Models for the Visual Cortexcited by 0
  13. 2025No-Regret Gaussian Process Optimization of Time-Varying Functionscited by 0
  14. 2026Separating Oblivious and Adaptive Models of Variable Selectioncited by 0
  15. 2026Informationally Compressive Anonymization: Non-Degrading Sensitive Input Protection for Privacy-Preserving Supervised Machine Learningcited by 0
  16. 2026On the Peril of (Even a Little) Nonstationarity in Satisficing Regret Minimizationcited by 0
  17. 2026On the Invariance and Generality of Neural Scaling Lawscited by 0
  18. 2026On the Structural Limits of Machine Learning Decision Systems: An Information-Theoretic, Interaction-Based, and Stochastic-Dynamical Perspectivecited by 0