# EQUI-VOCAL: Synthesizing Queries for Compositional Video Events from Limited User Interactions [Technical Report]

arXiv: https://arxiv.org/abs/2301.00929 · HTML: https://mlsanity.com/paper/2301.00929/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2023.

## Built on by (most cited first, 10 of 10)

- [Agile Modeling: From Concept to Classifier in Minutes](https://mlsanity.com/paper/2302.12948/) (2023)
- [Multi-Modal Data Exploration via Language Agents](https://mlsanity.com/paper/2412.18428/) (2024)
- [LazyVLM: Neuro-Symbolic Approach to Video Analytics](https://mlsanity.com/paper/2505.21459/) (2025)
- [Hydro: Adaptive Query Processing of ML Queries](https://mlsanity.com/paper/2403.14902/) (2024)
- [TRACER: Efficient Object Re-Identification in Networked Cameras through Adaptive Query Processing](https://mlsanity.com/paper/2507.09448/) (2025)
- [KathDB: Explainable Multimodal Database Management System with Human-AI Collaboration](https://mlsanity.com/paper/2512.11067/) (2025)
- [Human-Data Interaction, Exploration, and Visualization in the AI Era: Challenges and Opportunities](https://mlsanity.com/paper/2603.05542/) (2026)
- [Search over the Visual World: Persistent Visual Memory, Layered Indexes, and Source-Grounded Evidence](https://mlsanity.com/paper/2608.08075/) (2026)
- [KathDB-FAO: Synthesized Query Plans in a Multimodal DBMS](https://mlsanity.com/paper/2609.28761/) (2026)
- [Hob-VL: A Benchmark for Visually Grounded Boolean Reasoning](https://mlsanity.com/paper/2610.01605/) (2026)

## Abstract (arXiv)

We introduce EQUI-VOCAL: a new system that automatically synthesizes queries over videos from limited user interactions. The user only provides a handful of positive and negative examples of what they are looking for. EQUI-VOCAL utilizes these initial examples and additional ones collected through active learning to efficiently synthesize complex user queries. Our approach enables users to find events without database expertise, with limited labeling effort, and without declarative specifications or sketches. Core to EQUI-VOCAL's design is the use of spatio-temporal scene graphs in its data model and query language and a novel query synthesis approach that works on large and noisy video data. Our system outperforms two baseline systems -- in terms of F1 score, synthesis time, and robustness to noise -- and can flexibly synthesize complex queries that the baselines do not support.