# Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

arXiv: https://arxiv.org/abs/2606.16981 · HTML: https://mlsanity.com/paper/2606.16981/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 72 references that are in the map.
- First posted on arXiv 15 Jun 2026.

## Builds on (most cited first, 6 of 6)

- [Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations](https://mlsanity.com/paper/1606.01305/) (2016)
- [Synthesizing Credit Card Transactions](https://mlsanity.com/paper/1910.03033/) (2019)
- [Adaptive Threshold Sampling](https://mlsanity.com/paper/1708.04970/) (2017)
- [Stream Aggregation Through Order Sampling](https://mlsanity.com/paper/1703.02693/) (2017)
- [Railgun: managing large streaming windows under MAD requirements](https://mlsanity.com/paper/2106.12626/) (2021)
- [Locally Differentially Private Embedding Models in Distributed Fraud Prevention Systems](https://mlsanity.com/paper/2401.02450/) (2024)

## Abstract (arXiv)

Streaming data systems increasingly underpin Machine Learning workflows that maintain large numbers of continuously updated aggregations. In production settings, each incoming event typically triggers read-modify-write operations to persistent storage, making high-frequency state updates a dominant source of latency, contention, and operational cost. In this work, we decouple inference from state persistence in streaming Machine Learning pipelines via probabilistic thinning: every event is scored, but durable state updates are selectively triggered by informative events. Unlike approaches that shed input or state, we show that persistence-path control is achievable without a high-frequency in-memory control plane or cross-worker coordination, relying exclusively on approximate statistics retrieved from disk-backed key-value stores. We model the resulting stochastic processes, derive bounds on filtering rates, and prove that common time-based aggregations remain unbiased under variance-aware formulations, preventing systemic error accumulation. We evaluate the approach in a controlled setting that isolates per-event costs, demonstrating substantial reductions in storage Input/Output and serialization overhead. Across experiments, up to 90% of events are excluded from the persistence path while preserving and in some cases improving downstream utility.