CNS Core: Small: Not All Cameras are Created Equal: Systems Support for Highly Adaptive Video Analytics Pipelines

CNS 核心：小型：并非所有摄像机都是一样的：对高度自适应视频分析管道的系统支持

• 批准号: 2153449
• 负责人: Ravi Netravali
• 金额: $ 50万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153449&HistoricalAwards=false
• 关键词: CNS; Core; Small; Cameras; Created

The ubiquity of video camera deployments, coupled with steady improvements in computer vision algorithms, has given rise to a diverse range of video analytics applications. Use cases include surveillance, traffic scheduling, disaster response, and more. Yet despite their promise, video analytics deployments are far from widespread. A key reason is that video analysis is often prohibitively expensive: video is data-intensive, stressing the network, and analysis typically involves Deep Neural Networks (DNNs) to query video, requiring substantial compute resources. This project aims to design and implement practical video analytics systems that can adapt their execution to most efficiently utilize end-to-end compute and network resources, i.e., across cameras, servers, and the networks between them.The key insight underlying the proposed work is to adaptively place analytics tasks by leveraging frame-transforming techniques that are diverse in terms of resource requirements and accuracy, e.g., lightweight frame differencing versus expensive object detection DNNs. Along these lines, the project involves three synergistic directions. First, it rigorously classifies existing frame transforming techniques, investigating the correlation between their computation costs, potential data reduction, and impact on response accuracy. Second, it develops end-to-end systems that can automatically select the appropriate frame transforming technique to run on a camera with the goal of optimizing for response latency and accuracy given the available resources. Third, it develops techniques to extend adaptive video analytics to emerging camera settings, e.g., multi-camera, steerable, energy-harvesting; these systems rely on the extraction of spatial and temporal relationships between camera feeds to guide resource allocation decisions.The proposed research targets a large slice of the population (given the breadth of video analytics applications), and improves both the accessibility and potential of video analytics deployments. The developed systems enable affordable (but effective) video analytics for organizations of different scale, allowing them to make the most of their available resources. Furthermore, the work motivates novel applications that were previously deemed impractical, e.g., real-time monitoring of rural areas via energy-harvesting cameras. The project also involves outreach efforts to attract students from populations currently under-represented in computer science. Key to these efforts is magnifying the interdisciplinary nature of video analytics pipelines which span systems, networks, machine learning, and computer vision.The software and research artifacts designed as part of this project are released on a public website: http://web.cs.ucla.edu/~ravi/adaptive_video_analytics/. The site is regularly maintained and includes replication instructions and packages. Project data are kept on the site for at least 5 years after publication, with extensions based on public interest.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

摄像机部署的普遍性，加上计算机视觉算法的稳步改进，催生了各种各样的视频分析应用。用例包括监控、交通调度、灾难响应等。然而，尽管视频分析的前景广阔，但其部署还远未普及。一个关键原因是视频分析通常非常昂贵：视频是数据密集型的，给网络带来压力，并且分析通常涉及深度神经网络 (DNN) 来查询视频，需要大量的计算资源。该项目旨在设计和实现实用的视频分析系统，该系统可以调整其执行，以最有效地利用端到端计算和网络资源，即跨摄像机、服务器及其之间的网络。拟议工作的关键见解是通过利用在资源需求和准确性方面不同的帧转换技术来自适应地放置分析任务，例如，轻量级帧差分与昂贵的对象检测 DNN。沿着这些思路，该项目涉及三个协同方向。首先，它对现有的帧转换技术进行严格分类，研究其计算成本、潜在数据减少以及对响应准确性的影响之间的相关性。其次，它开发了端到端系统，可以自动选择适当的帧转换技术在相机上运行，​​目标是在给定可用资源的情况下优化响应延迟和准确性。第三，它开发了将自适应视频分析扩展到新兴摄像机设置的技术，例如多摄像机、可操纵、能量收集；这些系统依靠提取摄像机输入之间的空间和时间关系来指导资源分配决策。拟议的研究针对很大一部分人群（考虑到视频分析应用的广度），并提高视频分析的可访问性和潜力部署。开发的系统可为不同规模的组织提供经济实惠（但有效）的视频分析，使他们能够充分利用可用资源。此外，这项工作还激发了以前被认为不切实际的新颖应用，例如通过能量收集摄像头对农村地区进行实时监控。该项目还涉及外展工作，以吸引目前计算机科学领域代表性不足的人群中的学生。这些努力的关键是放大视频分析管道的跨学科性质，涵盖系统、网络、机器学习和计算机视觉。作为该项目一部分设计的软件和研究工件在公共网站上发布：http://web。 cs.ucla.edu/~ravi/adaptive_video_analytics/。该站点定期维护，并包含复制说明和包。项目数据在发布后在网站上保存至少 5 年，并根据公共利益进行延期。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics

RECL：视频分析的响应式资源高效持续学习

• 发表时间: 2023-04
• 期刊: 20th USENIX Symposium on Networked Systems Design and Implementation
• 作者: Khani, Mehrdad;Ananthanarayan, Ganesh;Hsieh, Kevin;Jiang, Junchen;Netravali, Ravi;Shu, Yuanchao;Alizadeh, Mohammad;Bahl, Victor
• 通讯作者: Bahl, Victor

Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics

博格特：迈向回顾性视频分析的通用加速

• 发表时间: 2021-06-21
• 作者: Neil Agarwal;R. Netravali
• 通讯作者: R. Netravali

Understanding the potential of server-driven edge video analytics

了解服务器驱动的边缘视频分析的潜力

• DOI: 10.1145/3508396.3512872
• 发表时间: 2022-01
• 期刊: HotMobile '22: Proceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications
• 作者: Zhang, Qizheng;Du, Kuntai;Neil Agarwal;Netravali, Ravi;Jiang, Junchen
• 通讯作者: Jiang, Junchen

Gemel: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge

Gemel：模型合并可在边缘进行内存高效、实时视频分析

• 发表时间: 2023-04
• 期刊: 20th USENIX Symposium on Networked Systems Design and Implementation
• 作者: Arthi Padmanabhan; Neil Agarwal
• 通讯作者: Neil Agarwal

Privid: Practical, Privacy-Preserving Video Analytics Queries

Privid：实用、保护隐私的视频分析查询

• DOI: 10.1049/el:19820656
• 发表时间: 2021-06-22
• 期刊: Electronics Letters
• 影响因子: 1.1
• 作者: Frank Cangialosi;Neil Agarwal;V. Arun;Junchen Jiang;Srinivas Narayana;An;D. Sarwate;Ravi Netravali
• 通讯作者: Ravi Netravali