CNS Core: Small: Software-Defined Video Analytics Pipeline: Enabling Resilient, High-Accuracy, and Resource-Effective Video Analytics

CNS 核心：小型：软件定义的视频分析管道：实现弹性、高精度和资源高效的视频分析

• 批准号: 2211459
• 负责人: Charlie Hu
• 金额: $ 43.81万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-15 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211459&HistoricalAwards=false
• 关键词: CNS; Core; Small; Software; Defined

Significant progress in machine learning and computer vision techniques along with growth in Internet of Things, edge computing and high-bandwidth access networks such as 5G in recent years have led to the wide adoption of video analytics systems. Such systems deploy cameras in major cities in the US and around the world to support diverse applications in surveillance, transportation, public safety, health-care, retail, and home automation. A typical video analytics system deployment consists of a video analytics pipeline (VAP), where video cameras are deployed at different locations of interest such as airports and hospitals to continuously capture video streams and transport them over the network (e.g., 5G) to the cloud servers that perform video analytics processing. As the network condition, compute resource availability, and importantly the content of the captured video frames undergo changes over time, the VAP needs to be continuously adapted in order to support resilient, high-accuracy and resource-efficient video analytics applications. The large amount of proposed VAP adaptation design in recent years ignore the built-in frame/video processing configurability of modern cameras, rely on costly offline/online profiling, and are limited to simple frame/video adaptations such as frame rate tuning and down-sampling. This project aims to develop key technologies that enable a software-defined video analytics pipeline architecture that supports resilient, high-accuracy, resource-efficient video analytics using commodity reconfigurable network cameras widely available in the market today. It will develop (1) the first software-defined VAP abstraction that instills “intelligence” into the very first stage of a video analytics pipeline, the camera itself, (2) the first software architecture that enables fully automated, real-time adaptation of VAPs by exploiting reconfigurable cameras, which has the potential to significantly improve the resilience of video analytics systems to environmental condition changes around the camera, and (3) the first capability to jointly adapt complex camera parameters to optimize the accuracy and resource usage of multiple analytics tasks that share a VAP and hence its camera capture, which lowers the cost of VAP deployment.The proposed research will have direct, practical implications to the video analytics industry and large societal impact. (1) The proposed software-defined VAP architecture will provide a much needed reference system design and implementation of high-accuracy, resource-efficient VAPs that maximally exploit the in-built frame processing capabilities of modern network cameras, and thus has the potential to foster the proliferation and wide adoption of “smart” cameras in video analytics system deployment. (2) The technologies developed for enabling resilient, high-accuracy, resource-efficient and cost-efficient VAPs will foster wide adoption of many important societal VAP applications such as transportation, entertainment, health-care, retail, automotive, home automation, safety, and security. (3) Technically, this work will have a far-reaching impact beyond the area of optimizing video analytics systems by developing general software-defined architectures for optimizing other classes of remote sensing systems and applications based on smart sensors such as LiDARs and UWB sensors. The research team will actively disseminate and transfer the technologies developed to the video analytics industry, and help organize the annual IEEE Autonomous Unmanned Aerial Vehicles (UAV) Competition for high school students world-wide.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习和计算机视觉技术的重大进展以及物联网，边缘计算和高带宽访问网络（例如5G）（例如5G）的增长，导致了视频分析系统的广泛采用。此类系统在美国和世界各地的主要城市部署摄像头，以支持监视，运输，公共安全，医疗保健，零售和家庭自动化方面的潜水员应用程序。一个典型的视频分析系统部署由视频分析管道（VAP）组成，在该管道中，摄像机被部署在不同感兴趣的不同位置，例如机场和医院，以连续捕获视频流并通过网络（例如5G）将其运输到执行视频分析处理的云服务器上。随着网络条件，计算资源可用性，重要的是，随着时间的推移，捕获的视频框架的内容发生变化，因此需要持续调整VAP，以支持弹性，高敏锐和资源有效的视频分析应用程序。近年来，大量提出的VAP适应设计忽略了现代相机的内置框架/视频处理配置，依赖于昂贵的离线/在线分析，并且仅限于简单的框架/视频改编，例如框架速率调整和下降。该项目旨在开发关键技术，以使其将开发软件定义的视频分析管道架构（1）（1）第一个软件定义的VAP抽象，将“智能”灌输到视频分析管道的第一阶段，相机本身，（2）可以完全自动化的vapsigration the vapsigrigigrigigrigigigigrigigigigrigigigigigrigigigrigigrigigigigigigrigigigigigigigigigigig，该镜头可实现该软件的范围。视频分析系统以环境状况周围的环境状况发生变化，以及（3）共同调整复杂摄像机参数的第一个能力，以优化多个分析任务的准确性和资源使用，这些任务具有共享VAP的多个分析任务，从而降低了VAP部署的成本。拟议的研究将导致对视频分析行业的实践影响，并影响了视频分析行业和大型社会效果。 （1）所提出的软件定义的VAP体系结构将提供急需的参考系统设计，并实现高准确性，资源有效的VAP，这些VAP最大程度地探索了现代网络摄像机的内部框架处理能力，因此有可能促进视频分析系统部署中“智能”摄像头的扩散和广泛采用。 （2）开发的技术是为了实现弹性，高临界性，资源效率和成本效益的VAP，将促进广泛采用许多重要的社会VAP应用，例如运输，娱乐，娱乐，医疗保健，零售，零售，汽车，汽车，自动化，自动化，安全，安全和安全。 （3）从技术上讲，这项工作将通过开发一般软件定义的体系结构来优化其他类别的远程传感器和应用程序，以基于LIDARS和UWB传感器等智能传感器来优化其他类别的远程传感器和应用程序。研究团队将积极传播和转移开发到视频分析行业的技术，并帮助组织全球高中学生的年度IEEE自动驾驶无人驾驶汽车（UAV）竞赛。这项奖项反映了NSF的法定任务，并通过评估该基金会的知识分子优点和广泛的影响来评估NSF的法定任务。

项目成果

APT: Adaptive Perceptual quality based camera Tuning using reinforcement learning

• DOI: 10.1109/iotsms58070.2022.10062226
• 发表时间: 2022-11
• 期刊: 2022 9th International Conference on Internet of Things: Systems, Management and Security (IOTSMS)
• 作者: Sibendu Paul;Kunal Rao;G. Coviello;Murugan Sankaradas;Oliver Po;Y. C. Hu;S. Chakradhar

Why is the video analytics accuracy fluctuating, and what can we do about it?

• DOI: 10.48550/arxiv.2208.12644
• 发表时间: 2022-08
• 作者: Sibendu Paul;Kunal Rao;G. Coviello;Murugan Sankaradas;Oliver Po;Y. C. Hu;S. Chakradhar

Enhancing Video Analytics Accuracy via Real-time Automated Camera Parameter Tuning

• DOI: 10.1145/3560905.3568527
• 发表时间: 2021-07
• 期刊: Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems
• 作者: Sibendu Paul;Kunal Rao;G. Coviello;Murugan Sankaradas;Oliver Po;Y. C. Hu;S. Chakradhar