CDSE: Collaborative: Cyber Infrastructure to Enable Computer Vision Applications at the Edge Using Automated Contextual Analysis

CDSE：协作：使用自动上下文分析在边缘启用计算机视觉应用的网络基础设施

• 批准号: 2104377
• 负责人: Vipin Chaudhary
• 金额: $ 10万
• 依托单位: Case Western Reserve University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2104377&HistoricalAwards=false
• 关键词: CDSE; Collaborative; Cyber; Infrastructure; Enable; CDSE; Collaborative; Cyber; Infrastructure; Enable

Digital cameras are deployed as network edge devices, gathering visual data for such tasks as autonomous driving, traffic analysis, and wildlife observation. Analyzing the vast amount of visual data is a challenge. Existing computer vision methods require fast computers that are beyond the computational capabilities of many edge devices. This project aims to improve the efficiency of computer vision methods so that they can run on battery-powered edge devices. Based on the visual data and complementary metadata (e.g., geographical location, local time), the project first extracts contextual information (such as a city street is expected to be busy at rush hour). The contextual information can help assist determine whether analysis results are correct. For example, a wild animal is not expected on a city street. Moreover, contextual information can improve efficiency. Only certain pixels need to be analyzed (pixels on the road are useful for detecting cars, while pixels in the sky are not) and this can significantly reduce the amount of computation, thus enabling analysis on edge devices. This project constructs a cyberinfrastructure for three services: (1) understand contextual information to reduce the search space of analysis methods, (2) reduce computation by considering only necessary pixels, and (3) automate evaluation of analysis results based on the contextual information without human effort.Understanding contextual information is achieved by using background segmentation, GPS-location-dependent logic, and image depth maps. Background analysis leverages semantic segmentation and analysis over time to identify the background pixels and then generate inference rules via a background-implies-foreground relationship. If a pixel is consistently marked by the same semantic label across a long period of time, this pixel is classified as a background pixel. The background information can infer certain types of foreground objects. For example, if the background is city streets, the foreground objects can be vehicles or pedestrians; if a bison is detected, this is likely a mistake. This project processes only the foreground pixels by adding masks to the neural network layers. Masking convolution can substantially reduce the amount of computation with no loss of accuracy and no additional training is needed. Meanwhile, hierarchical neural networks can skip sections of a model based on context. For example, pixels in the sky only need to be processed by the hierarchy nodes that classify airplanes. The project provides an online service that can accept input data and analysis programs for automatic evaluation of the programs, without human created labels. The evaluation is based on the correlations of background and foreground objects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数码相机被部署为网络边缘设备，收集视觉数据，以进行自动驾驶，交通分析和野生动植物观察等任务。分析大量视觉数据是一个挑战。现有的计算机视觉方法需要快速计算机，这些计算机超出了许多边缘设备的计算功能。该项目旨在提高计算机视觉方法的效率，以便它们可以在电池供电的边缘设备上运行。根据视觉数据和互补的元数据（例如，地理位置，当地时间），该项目首先提取上下文信息（例如，城市街有望在高峰时段忙碌）。上下文信息可以帮助确定分析结果是否正确。例如，预计在城市街上不会发生野生动物。此外，上下文信息可以提高效率。 仅需要分析某些像素（道路上的像素可用于检测汽车，而天空中的像素不是），这可以大大减少计算量，从而在边缘设备上进行分析。 This project constructs a cyberinfrastructure for three services: (1) understand contextual information to reduce the search space of analysis methods, (2) reduce computation by considering only necessary pixels, and (3) automate evaluation of analysis results based on the contextual information without human effort.Understanding contextual information is achieved by using background segmentation, GPS-location-dependent logic, and image depth maps. 背景分析利用语义细分和分析随着时间的流逝，以识别背景像素，然后通过背景进化 - 前景关系生成推理规则。如果像素在很长一段时间内一直以相同的语义标签标记，则该像素被归类为背景像素。背景信息可以推断某些类型的前景对象。例如，如果背景是城市街道，前景物体可以是车辆或行人。如果检测到野牛，这可能是一个错误。该项目仅通过在神经网络层中添加口罩来处理前景像素。掩盖卷积可以大大减少计算量，而无需损失准确性，也不需要额外的训练。同时，分层神经网络可以根据上下文跳过模型的部分。例如，天空中的像素只需要通过对飞机进行分类的层次结构节点处理。该项目提供了一项在线服务，可以在没有人类创建的标签的情况下接受输入数据和分析程序，以自动评估程序。该评估基于背景和前景对象的相关性。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准来评估值得支持的。