A Domain-Specific Language for LLM-Driven Trigger Generation in Multimodal Data Collection
Not on the map. Why?
Abstract (arXiv)
Data-driven systems depend on task-relevant data, yet data collection pipelines remain passive and indiscriminate. Continuous logging of multimodal sensor streams incurs high storage costs and captures irrelevant data. This paper proposes a declarative framework for intent-driven, on-device data collection that enables selective collection of multimodal sensor data based on high-level user requests. The framework combines natural language interaction with a formally specified domain-specific language (DSL). Large language models translate user-defined requirements into verifiable and composable DSL programs that define conditional triggers across heterogeneous sensors, including cameras, LiDAR, and system telemetry. Empirical evaluation on vehicular and robotic perception tasks shows that the DSL-based approach achieves higher generation consistency and lower execution latency than unconstrained code generation while maintaining comparable detection performance. The structured abstraction supports modular trigger composition and concurrent deployment on resource-constrained edge platforms. This approach replaces passive logging with a verifiable, intent-driven mechanism for multimodal data collection in real-time systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 18 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
10 of 18 references.
References · 10
- 2025Gemma 3 Technical Reportcited by 1,824
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
- 2022Anomaly Detection in Autonomous Driving: A Surveycited by 87
- 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 58
- 2021An Application-Driven Conceptualization of Corner Cases for Perception in Highly Automated Drivingcited by 33
- 2023Unveiling Objects with SOLA: An Annotation-Free Image Search on the Object Level for Automotive Data Setscited by 5
- 2025A Framework for a Capability-driven Evaluation of Scenario Understanding for Multimodal Large Language Models in Autonomous Drivingcited by 5
- 2025Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selectioncited by 3
- 2025A Data-Driven Novelty Score for Diverse In-Vehicle Data Recordingcited by 3
Built on by
Not cited yet.