Leveraging FPGAs for Machine Learning Implementation and Acceleration

利用 FPGA 实现机器学习和加速

• 批准号: RGPIN-2020-07118
• 负责人: Brown, Stephen
• 金额: $ 2.84万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=718699
• 关键词: Leveraging; FPGAs; Machine; Learning; Implementation

This research program proposes to develop new technology that will lead to improved object detection and computer vision for autonomous vehicles using machine learning. The technology will include both software and hardware for compute-acceleration using field-programmable gate arrays (FPGAs). FPGAs are a type of integrated circuit chip that can be programmed to implement different applications in hardware. The program comprises three related research thrusts: neural-network (NN) algorithms that incorporate sensor-fusion techniques, NN circuit architectures for FPGAs, and end-user applications of this technology, such as enhancing vision for auto-drivers in conditions that are difficult for those drivers to manage. The traditional approach to object detection for autonomous vehicles has been a modified image classification task, which locates objects (e.g. cars, pedestrians) in a frame as well as performing an object classification task. However, as research has progressed, promising results with Light Detection and Ranging (LiDAR) data has catalyzed research into fusion of LiDAR and vision (camera) sensors. In this research program we propose to explore and fully understand fusion operations like the element-wise mean, but taking into consideration the predicted performance of individual sensors in novel ways. The performance evaluation of sensors becomes useful when one sensor is compromised through weather, component breakdown or intrusion attempts. When a sensor is incorrectly making predictions, the fusion-prediction network must be able to dynamically compensate to avoid degradation of the entire object detection system. State-of-the-art computer vision solutions for autonomous vehicles increasing rely on convolutional neural networks (CNNs) to achieve good quality of result. A CNN typically has millions of neurons and synapses which incur high computational complexity and storage requirements. Therefore, deploying CNNs on autonomous cars and drones as object detectors and sensor fusion solutions is difficult because of the tight power budget, low latency requirement, and scarce computation resources in these platforms. FPGAs have emerged as a popular substrate for implementing dedicated CNN accelerators for these applications. Results from this research program will be applied to a multi-university international self-driving automotive competition being sponsored by General Motors. Our focus will be on enhancing an automobile's ability to "see" properly in imperfect conditions, such as night driving, rain driving or snow driving. This technology could also be useful for any person who struggles with vision issues, including the elderly. Enhanced images could be provided through an existing wearable display, a clear display, via projection onto a windshield, and so on. Any person who suffers from vision restrictions could benefit greatly.

该研究计划旨在开发新技术，利用机器学习改进自动驾驶汽车的物体检测和计算机视觉。该技术将包括使用现场可编程门阵列（FPGA）进行计算加速的软件和硬件。 FPGA 是一种集成电路芯片，可以通过编程在硬件中实现不同的应用。该计划包括三个相关的研究重点：结合传感器融合技术的神经网络 (NN) 算法、FPGA 的 NN 电路架构以及该技术的最终用户应用，例如增强自动驾驶者在困难条件下的视力供那些司机管理。 自动驾驶车辆目标检测的传统方法是改进的图像分类任务，它定位帧中的目标（例如汽车、行人）并执行目标分类任务。然而，随着研究的进展，光探测和测距 (LiDAR) 数据的有希望的结果促进了 LiDAR 和视觉（相机）传感器融合的研究。在这个研究项目中，我们建议探索并充分理解融合操作，例如元素均值，但以新颖的方式考虑单个传感器的预测性能。当一个传感器因天气、组件故障或入侵尝试而受到损害时，传感器的性能评估就变得有用。当传感器错误地做出预测时，融合预测网络必须能够动态补偿，以避免整个物体检测系统性能下降。 最先进的自动驾驶汽车计算机视觉解决方案越来越依赖卷积神经网络 (CNN) 来获得高质量的结果。 CNN 通常具有数百万个神经元和突触，这会带来很高的计算复杂性和存储要求。因此，在自动驾驶汽车和无人机上部署 CNN 作为目标检测器和传感器融合解决方案是很困难的，因为这些平台的功耗预算紧张、延迟要求低且计算资源稀缺。 FPGA 已成为为这些应用实现专用 CNN 加速器的流行基板。 该研究项目的结果将应用于由通用汽车赞助的多所大学国际自动驾驶汽车竞赛。我们的重点是增强汽车在不完美的条件下（例如夜间驾驶、雨天驾驶或雪地驾驶）正确“看见”的能力。这项技术对于任何患有视力问题的人（包括老年人）也很有用。增强的图像可以通过现有的可穿戴显示器、清晰的显示器、通过投影到挡风玻璃上等来提供。任何患有视力障碍的人都可以受益匪浅。