mlsanityData to 14 Oct 2026

Prediction-Oriented Subsampling from Data Streams

arXiv 2508.03868 ↗First posted 5 Aug 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Data is often generated in streams, with new observations arriving over time. A key challenge for learning models from data streams is capturing relevant information while keeping computational costs manageable. We explore intelligent data subsampling for offline learning, and argue for an information-theoretic method centred on reducing uncertainty in downstream predictions of interest. Empirically, we demonstrate that this prediction-oriented approach performs better than a previously proposed information-theoretic technique on two widely studied problems. At the same time, we highlight that reliably achieving strong performance in practice requires careful model design.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 57 references that are in the map. First posted on arXiv 5 Aug 2025.

Builds on

34 of 57 references.

References · 34

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  5. 2018Experience Replay for Continual Learningcited by 1,095
  6. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  7. 2019Gradient based sample selection for online continual learningcited by 571
  8. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  9. 2017Variational Continual Learningcited by 339
  10. 2019On Tiny Episodic Memories in Continual Learningcited by 307
  11. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  12. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 201
22 more
  1. 2013A General Framework for Updating Belief Distributionscited by 175
  2. 2023Modern Bayesian Experimental Designcited by 125
  3. 2021Epistemic Neural Networkscited by 120
  4. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 117
  5. 2020Continual Deep Learning by Functional Regularisation of Memorable Pastcited by 105
  6. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 93
  7. 2024Continual Learning and Catastrophic Forgettingcited by 77
  8. 2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 75
  9. 2023Prediction-Oriented Bayesian Active Learningcited by 62
  10. 2022Information-theoretic Online Memory Selection for Continual Learningcited by 42
  11. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 42
  12. 2021Conformal Bayesian Computationcited by 39
  13. 2021From Predictions to Decisions: The Importance of Joint Predictive Distributionscited by 29
  14. 2023Continual Learning via Sequential Function-Space Variational Inferencecited by 29
  15. 2022Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantitiescited by 28
  16. 2021The Neural Testbed: Evaluating Joint Predictionscited by 22
  17. 2022Fine-Tuning Language Models via Epistemic Neural Networkscited by 22
  18. 2020Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?cited by 21
  19. 2024Random Representations Outperform Online Continually Learned Representationscited by 11
  20. 2014Approximate Models and Robust Decisionscited by 10
  21. 2022Evaluating High-Order Predictive Distributions in Deep Learningcited by 10
  22. 2024Making Better Use of Unlabelled Data in Bayesian Active Learningcited by 10

Built on by

Not cited yet.