OmniSense: Towards Edge-Assisted Online Analytics for 360-Degree Videos

With the reduced hardware costs of omnidirectional cameras and the proliferation of various extended reality applications, more and more 360^circ videos are being captured. To fully unleash their potential, advanced video analytics is expected to extract actionable insights and situational knowledge without blind spots from the videos. In this paper, we present OmniSense, a novel edge-assisted framework for online immersive video analytics. OmniSense achieves both low latency and high accuracy, combating the significant computation and network resource challenges of analyzing 360^circ videos. Motivated by our measurement insights into 360^circ videos, OmniSense introduces a lightweight spherical region of interest (SRoI) prediction algorithm to prune redundant information in 360^circ frames. Incorporating the video content and network dynamics, it then smartly scales vision models to analyze the predicted SRoIs with optimized resource utilization. We implement a prototype of OmniSense with commodity devices and evaluate it on diverse real-world collected 360^circ videos. Extensive evaluation results show that compared to resource-agnostic baselines, it improves the accuracy by 19.8% -- 114.6% with similar end-to-end latencies. Meanwhile, it hits 2.0× -- 2.4× speedups while keeping the accuracy on par with the highest accuracy of baselines.