Collaborative Research: Enabling Machine Learning based Cooperative Perception with mmWave Communication for Autonomous Vehicle Safety

协作研究：通过毫米波通信实现基于机器学习的协作感知，以实现自动驾驶汽车安全

• 批准号: 2010366
• 负责人: Hua Fang
• 金额: $ 15.33万
• 依托单位: University of Massachusetts, Dartmouth
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-15 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2010366&HistoricalAwards=false
• 关键词: Collaborative; Research; Enabling; Machine; Learning

By understanding what and how data are exchanged among autonomous vehicles, from a machine learning perspective, it is possible to realize precise cooperative perception on autonomous vehicles, enabling massive amounts of sensor information to be shared amongst vehicles. Such an advance can be extremely useful to extend the line of sight and field of view of autonomous vehicles, which otherwise suffers from blind spots and occlusions. The extended field of view on autonomous vehicles will be beneficial at times when there are occlusions preventing a complete perception of the environment. This increase in situational awareness promotes safe driving over a narrow scope and improves traffic flow efficiency over an extended scope. The proposed research work will not only change the way people think about the perception system on autonomous vehicles but could also open up opportunities to design novel systems that were previously inconceivable. This project offers a wide variety of research activities from data collection, algorithm design, system development, and in-the-field evaluation, which will be attractive to students with various backgrounds and goals. Undergraduate and graduate students will be involved directly in the research activities as assistants at different levels. The expected research outcomes from this project will also enhance the current curricula related to machine learning, Internet of things, and wireless communications.The main research objective of this project is to understand the sensing and communication challenges to achieving cooperative perception among autonomous vehicles, and to use the insights thus gained to guide the design of suitable data exchange format, data fusion algorithms, and efficient millimeter wave vehicular communications. Results from this project will include a machine learning based cooperative perception framework, which will shed light on effectively combining feature maps, derived from machine learning models on autonomous vehicles, in a distributed manner. The resulted feature map compression and feature map selection approaches will significantly reduce the amount of data exchanged among vehicles, enabling agile and precise cooperative perception on connected and autonomous vehicles. The proposed scalable feature map transmission mechanism jointly considers the application requirements, link and physical layer characteristics of millimeter wave links, enabling sensor data sharing on a massive scale among autonomous vehicles. The implemented system and evaluation platform will serve as a convincing proof-of-concept for the proposed solution, thus opening the door to widespread adoption of cooperative perception applications via millimeter wave communications in future vehicle networks. The collected dataset from this project will be made publicly available, serving as a catalyst for enabling innovative research on cooperative object detection, vehicular edge computing, and machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从机器学习的角度来看，通过了解自动驾驶汽车之间的数据以及如何交换，可以实现对自动驾驶汽车的精确合作感，从而使大量的传感器信息在车辆中共享。这样的进步对于扩展自动驾驶汽车的视线和视野非常有用，否则会遭受盲点和遮挡。自动驾驶汽车的扩展视野有时会在阻止环境完全看法的情况下是有益的。这种情境意识的这种提高可促进在狭窄的范围内安全驾驶，并提高延长范围的交通流量效率。拟议的研究工作不仅会改变人们对自动驾驶汽车的感知系统的思考方式，而且还可以为设计以前无法想象的新型系统打开机会。该项目提供了各种各样的研究活动，包括数据收集，算法设计，系统开发和现场评估，这将对具有各种背景和目标的学生有吸引力。本科生和研究生将直接作为不同级别的助手参与研究活动。该项目的预期研究结果还将增强与机器学习，物联网和无线通信相关的当前课程。该项目的主要研究目标是了解实现自动驾驶汽车之间合作感的感应和沟通挑战，并使用洞察力，从而使用洞察力，从而使用适当的数据交换格式，数据融合量且有效地播放了适当的数据交换格式，并有效地浪费了量。该项目的结果将包括一个基于机器学习的合作感知框架，该框架将以分布式方式阐明从自动驾驶汽车上的机器学习模型中得出的特征图。所得的特征图压缩和特征图选择方法将显着减少车辆之间交换的数据量，从而对连接和自动驾驶汽车产生敏捷和精确的合作感。提出的可扩展特征图传输机制共同考虑了毫米波链路的应用要求，链接和物理层特征，从而使传感器数据在自动驾驶汽车之间大规模分享。实施的系统和评估平台将作为拟议解决方案的令人信服的概念验证，从而为通过未来车辆网络中的毫米波通信提供广泛采用合作感知应用的大门。该项目中收集的数据集将公开可用，可作为催化剂，以促进有关合作对象检测，车辆边缘计算和机器学习的创新研究。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力和更广泛影响的评估来通过评估来获得支持的。

项目成果

Adaptive Vehicle Platooning with Joint Network-Traffic Approach

• DOI: 10.1109/globecom46510.2021.9685116
• 发表时间: 2021-12
• 期刊: 2021 IEEE Global Communications Conference (GLOBECOM)
• 作者: Chinmay Mahabal;Hua Fang;Honggang Wang;Qing Yang

A Survey of Collaborative Machine Learning Using 5G Vehicular Communications

• DOI: 10.1109/comst.2022.3149714
• 发表时间: 2022-01-01
• 期刊: IEEE COMMUNICATIONS SURVEYS AND TUTORIALS
• 影响因子: 35.6
• 作者: Balkus, Salvador, V;Wang, Honggang;Fang, Hua
• 通讯作者: Fang, Hua

Machine-Learning-Enabled Cooperative Perception for Connected Autonomous Vehicles: Challenges and Opportunities

• DOI: 10.1109/mnet.011.2000560
• 发表时间: 2021-05
• 期刊: IEEE Network
• 影响因子: 9.3
• 作者: Qing Yang;Song Fu;Honggang Wang;Hua Fang

Dual Mode Localization Assisted Beamforming for mmWave V2V Communication

• DOI: 10.1109/tvt.2022.3175165
• 发表时间: 2022-09-01
• 期刊: IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
• 影响因子: 6.8
• 作者: Mahabal, Chinmay;Wang, Honggang;Fang, Hua
• 通讯作者: Fang, Hua

Beamforming and Scalable Image Processing in Vehicle-to-Vehicle Networks

车对车网络中的波束成形和可扩展图像处理

• DOI: 10.1007/s11265-021-01696-6
• 发表时间: 2022
• 期刊: Journal of Signal Processing Systems
• 作者: Ngo, Hieu;Fang, Hua;Wang, Honggang
• 通讯作者: Wang, Honggang