CPS: Medium: GOALI: Real-Time Computer Vision in Autonomous Vehicles: Real Fast Isn't Good Enough

CPS：中：GOALI：自动驾驶汽车中的实时计算机视觉：真正的快还不够好

• 批准号: 1837337
• 负责人: James Anderson
• 金额: $ 100万
• 依托单位: University of North Carolina at Chapel Hill
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1837337&HistoricalAwards=false
• 关键词: CPS; Medium; GOALI; Real; Time

The push towards deploying autonomous-driving capabilities in vehicles is happening at breakneck speed. Semi-autonomous features are becoming increasingly common, and fully autonomous vehicles at mass-market scales are on the horizon. Cameras are cost-effective sensors, so computer-vision techniques have loomed large in implementing autonomous features. In a vehicle, these techniques must function "in real time." Unfortunately, this requirement lies at the heart of a significant disconnect: when computer-vision researchers refer to "real time," they usually mean "real fast"; in contrast, certifiable automotive systems must be "real time" in the sense of being able to predictably react to input information (such as a detected pedestrian) within specified deadlines so that adverse outcomes (such as striking a pedestrian) are provably precluded. The goal of this project is to eliminate this disconnect. It will do so through research on several fronts. First, a real-time computer-vision programming framework will be created by extending OpenVX, which is a recently ratified standard intended for developing computer-vision applications for embedded systems. Second, new computer-vision algorithms that exploit the features of this programming framework will be created, and methods will be developed to transform existing algorithms to make them "real time" in a predictability sense. Third, an experimental evaluation of "real-fast" vs. "real-time" computer vision will be conducted using driving simulators, sub-scale autonomous vehicles, and advanced testing infrastructure at General Motors.While industry is pushing hard in the area of autonomous driving, autonomous vehicles will never become a common mode of transportation unless methods for certifying real-time safety are produced. This project will focus on a key aspect of certification: validating the real-time correctness of computer-vision applications. The results that are produced will be made available to the world at large through open-source software. This software will include the new programming framework to be produced as well as tools for validating the real-time correctness of applications developed using this framework. In this project, a special emphasis will be placed on outreach to girls and women, as three female graduate students will be involved in the project. Such outreach will include: events involving the Graduate Women in Computer Science (GWiCS) group at the University of North Carolina (UNC), which hosts an annual research symposium targeted toward undergraduate women and other under-represented minorities; Tar Heel Hack, a hackathon for local middle and high school girls; the UNC Girls Who Code Club, which provides local girls in grades 6-12 with a community for learning about computer science; and the UNC Computer Science Department's annual Open House and Science Expo. These events will include hackathon projects as well as demos of a driving simulator and a sub-scale autonomous car.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在车辆中部署自动驾驶功能的推动正在以惊人的速度进行。 半自动驾驶功能正变得越来越普遍，大众市场规模的全自动驾驶汽车也即将问世。 相机是具有成本效益的传感器，因此计算机视觉技术在实现自主功能方面发挥着重要作用。 在车辆中，这些技术必须“实时”发挥作用。 不幸的是，这个要求是一个重大脱节的核心：当计算机视觉研究人员提到“实时”时，他们通常意味着“真正快”；相比之下，可认证的汽车系统必须是“实时”的，即能够在指定的期限内对输入信息（例如检测到的行人）做出可预测的反应，从而可以证明避免不良后果（例如撞到行人）。 该项目的目标是消除这种脱节。 它将通过几个方面的研究来实现这一目标。 首先，将通过扩展 OpenVX 创建实时计算机视觉编程框架，OpenVX 是最近批准的标准，旨在开发嵌入式系统的计算机视觉应用程序。 其次，将创建利用该编程框架特征的新计算机视觉算法，并将开发方法来改造现有算法，使其在可预测的意义上实现“实时”。 第三，将使用通用汽车的驾驶模拟器、小型自动驾驶汽车和先进的测试基础设施对“实时”计算机视觉与“实时”计算机视觉进行实验评估。自动驾驶，除非产生证明实时安全的方法，否则自动驾驶汽车永远不会成为一种常见的交通方式。 该项目将重点关注认证的一个关键方面：验证计算机视觉应用程序的实时正确性。 产生的结果将通过开源软件向全世界公开。 该软件将包括要生成的新编程框架以及用于验证使用该框架开发的应用程序的实时正确性的工具。 在该项目中，将特别重视对女孩和妇女的宣传，因为三名女研究生将参与该项目。 此类外展活动将包括： 涉及北卡罗来纳大学 (UNC) 计算机科学研究生女性 (GWiCS) 小组的活动，该小组每年举办一次针对本科女性和其他代表性不足的少数群体的研究研讨会； Tar Heel Hack，针对当地初高中女生的黑客马拉松； UNC Girls Who Code Club，为当地 6-12 年级的女孩提供学习计算机科学的社区；以及北卡罗来纳大学计算机科学系的年度开放日和科学博览会。 这些活动将包括黑客马拉松项目以及驾驶模拟器和小型自动驾驶汽车的演示。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Exploiting Simultaneous Multithreading in Priority-Driven Hard Real-Time Systems

在优先级驱动的硬实时系统中利用同步多线程

• DOI: 10.1109/rtcsa50079.2020.9203575
• 发表时间: 2020
• 期刊: Proceedings of the 26th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications
• 作者: Osborne, Sims Hill;Ahmed, Shareef;Nandi, Saujas;Anderson, James H.
• 通讯作者: Anderson, James H.

Hardware Compute Partitioning on NVIDIA GPUs*

NVIDIA GPU 上的硬件计算分区*

• DOI: 10.1109/rtas58335.2023.00012
• 发表时间: 2023
• 期刊: Proceedings of the 29th IEEE Real-Time and Embedded Technology and Applications Symposium
• 作者: Bakita, Joshua;Anderson, James H.
• 通讯作者: Anderson, James H.

Soft Real-Time Gang Scheduling

软实时组调度

• DOI: 10.1109/rtss59052.2023.00036
• 发表时间: 2023
• 期刊: Proceedings of the 44th IEEE Real-Time Systems Symposium
• 作者: Ahmed, Shareef;Anderson, James H.
• 通讯作者: Anderson, James H.

Minimizing DAG Utilization by Exploiting SMT

• DOI: 10.1109/rtas54340.2022.00029
• 发表时间: 2022-05
• 期刊: 2022 IEEE 28th Real-Time and Embedded Technology and Applications Symposium (RTAS)
• 作者: S. Osborne;Joshua Bakita;Jingyuan Chen;Tyler Yandrofski;James H. Anderson

Optimal soft real-time semi-partitioned scheduling made simple (and dynamic)

• DOI: 10.1145/3356401.3356402
• 发表时间: 2019-11
• 期刊: Proceedings of the 27th International Conference on Real-Time Networks and Systems
• 作者: Clara Hobbs;Zelin Tong;James H. Anderson