# SketchQL Demonstration: Zero-shot Video Moment Querying with Sketches

arXiv: https://arxiv.org/abs/2405.18334 · HTML: https://mlsanity.com/paper/2405.18334/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 6 of 6)

- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [A Survey on Video Moment Localization](https://mlsanity.com/paper/2306.07515/) (2023)
- [Demonstration of Panda: A Weakly Supervised Entity Matching System](https://mlsanity.com/paper/2106.10821/) (2021)

## Built on by (most cited first, 1 of 1)

- [Human-Data Interaction, Exploration, and Visualization in the AI Era: Challenges and Opportunities](https://mlsanity.com/paper/2603.05542/) (2026)

## Abstract (arXiv)

In this paper, we will present SketchQL, a video database management system (VDBMS) for retrieving video moments with a sketch-based query interface. This novel interface allows users to specify object trajectory events with simple mouse drag-and-drop operations. Users can use trajectories of single objects as building blocks to compose complex events. Using a pre-trained model that encodes trajectory similarity, SketchQL achieves zero-shot video moments retrieval by performing similarity searches over the video to identify clips that are the most similar to the visual query. In this demonstration, we introduce the graphic user interface of SketchQL and detail its functionalities and interaction mechanisms. We also demonstrate the end-to-end usage of SketchQL from query composition to video moments retrieval using real-world scenarios.