# MacroBase: Prioritizing Attention in Fast Data

arXiv: https://arxiv.org/abs/1603.00567 · HTML: https://mlsanity.com/paper/1603.00567/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2016.

## Built on by (most cited first, 15 of 15)

- [Data-driven Discovery with Large Generative Models](https://mlsanity.com/paper/2402.13610/) (2024)
- [AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise](https://mlsanity.com/paper/2507.00310/) (2025)
- [DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications](https://mlsanity.com/paper/2109.11495/) (2021)
- [Learn the Time to Learn: Replay Scheduling in Continual Learning](https://mlsanity.com/paper/2209.08660/) (2022)
- [XInsight: eXplainable Data Analysis Through The Lens of Causality](https://mlsanity.com/paper/2207.12718/) (2022)
- [A Meta-level Analysis of Online Anomaly Detectors](https://mlsanity.com/paper/2209.05899/) (2022)
- [VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysis](https://mlsanity.com/paper/2505.03132/) (2025)
- [Explaining Inference Queries with Bayesian Optimization](https://mlsanity.com/paper/2102.05308/) (2021)
- [Efficient Dynamic Clustering: Capturing Patterns from Historical Cluster Evolution](https://mlsanity.com/paper/2203.00812/) (2022)
- [Explainable multi-class anomaly detection on functional data](https://mlsanity.com/paper/2205.02935/) (2022)
- [Interpretable Outlier Summarization](https://mlsanity.com/paper/2303.06261/) (2023)
- [MaskSearch: Querying Image Masks at Scale](https://mlsanity.com/paper/2305.02375/) (2023)
- [Towards Causal Physical Error Discovery in Video Analytics Systems](https://mlsanity.com/paper/2405.17686/) (2024)
- [RankShift: In-Database Detection and Explanation of Categorical Shifts](https://mlsanity.com/paper/2608.28922/) (2026)
- [Detect, Explain, Interpret: An End-to-End Benchmark for Time Series Anomaly Detection, Explainability and Interpretability](https://mlsanity.com/paper/2610.01168/) (2026)

## Abstract (arXiv)

As data volumes continue to rise, manual inspection is becoming increasingly untenable. In response, we present MacroBase, a data analytics engine that prioritizes end-user attention in high-volume fast data streams. MacroBase enables efficient, accurate, and modular analyses that highlight and aggregate important and unusual behavior, acting as a search engine for fast data. MacroBase is able to deliver order-of-magnitude speedups over alternatives by optimizing the combination of explanation and classification tasks and by leveraging a new reservoir sampler and heavy-hitters sketch specialized for fast data streams. As a result, MacroBase delivers accurate results at speeds of up to 2M events per second per query on a single core. The system has delivered meaningful results in production, including at a telematics company monitoring hundreds of thousands of vehicles.