CAREER: Modeling Situated Intention during Nondeterministic Pedestrian-Vehicle Interactions through Explainable Compositional Learning of Naturalistic Driving Data

职业：通过自然驾驶数据的可解释组合学习，对非确定性行人-车辆交互过程中的情境意图进行建模

• 批准号: 2145565
• 负责人: Renran Tian
• 金额: $ 59.81万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145565&HistoricalAwards=false
• 关键词: CAREER; Modeling; Situated; Intention; during

With the fast progress of artificial intelligence, vehicles with higher levels of automation are entering daily life. Automated driving technologies are expected to improve traffic safety, promote travel efficiency, protect the environment, reduce mobility barriers for older generations or people with disabilities, and thus deliver overall societal benefits. However, existing algorithms embedded in autonomous vehicles still face fundamental challenges in recognizing the quickly changing intentions of pedestrians moving on the road and sidewalks, making it hard to predict their behavior and plan vehicle motions. Such limitations impede the implementation of fully autonomous and safe cars in city environments and create additional risks for pedestrians and other road users. This project focuses on developing novel techniques to model and predict the complex and changing intentions of pedestrians. By learning the thinking process of drivers and their driving responses, an algorithm will be created to equip the automated cars with similar capabilities to interact with pedestrians and other road users smoothly and safely. The research process will include naturalistic driving data collection, subject experiments, knowledge modeling, and learning algorithm development. The developed algorithm will be evaluated in an immersive virtual environment. The project also includes activities to promote user-centered design in engineering education, foster the awareness of biases and ethical issues related to artificial intelligence technologies, and increase the participation of underrepresented communities in Science, Technology, Engineering, and Mathematics. This project surmounts limitations of current pedestrian behavior prediction to achieving mutual intelligibility between autonomous vehicles and pedestrians. Principally, unlike the traditional static view of pedestrian intention at a critical moment, this research investigates the relationship between non-verbal actions and intention changes of pedestrians moment-to-moment in dynamic (changing) and interactive situations. The project collects temporal video segments and human reasoning descriptions simultaneously through event-segmentation-based video experiments. Then, it develops a compositional learning method to learn and combine language features with visual features. This method can avoid the rigid structure of expert-selected feature space by creating collective features from ordinary drivers, and the three-level explainability of the learning model can justify model outputs from input features. Finally, the developed intention prediction model will be evaluated through subject experiments in a virtual interactive pedestrian simulator. The research findings will be shared through industrial collaborators and conferences, and a pedestrian behavior benchmark dataset will be disseminated to the public. The research results will be included in engineering education to promote design approaches that take into account the users’ feelings, values, and overall mental state (empathic design). These educational activities will include courses at the investigator’s university, the autonomous driving research community, and industry. They will increase the awareness of critical human-centered AI issues like biases, trust, and social intelligence in design of AI.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着人工智能的快速进步，自动化程度更高的车辆正在走进日常生活，自动驾驶技术有望改善交通安全、提高出行效率、保护环境、减少老一辈或残疾人的出行障碍。然而，自动驾驶汽车中嵌入的现有算法在识别道路和人行道上移动的行人的快速变化的意图方面仍然面临着根本性的挑战，这使得预测他们的行为和规划车辆运动变得困难，这种限制阻碍了自动驾驶汽车的实施。城市环境中完全自主且安全的汽车，并创造该项目的重点是开发新技术来建模和预测行人复杂且不断变化的意图，通过学习驾驶员的思维过程及其驾驶反应，将创建一种算法来装备自动驾驶汽车。具有与行人和其他道路使用者顺利、安全地互动的类似功能，研究过程将包括自然驾驶数据收集、主题实验、知识建模和学习算法开发，并将在沉浸式虚拟环境中进行评估。还包括促进以用户为中心的设计的活动该项目克服了当前行人行为预测的局限性，实现了自动驾驶汽车之间的相互理解。与行人在关键时刻的意图的传统静态观点不同，本研究主要研究动态（变化）和交互情境中行人的非语言行为与意图变化之间的关系。通过基于事件分割的视频实验同时收集时间视频片段和人类推理描述，然后，它开发了一种组合学习方法来学习并将语言特征与视觉特征相结合，该方法可以避免专家选择的特征空间的刚性结构。创建普通驾驶员的集体特征，并且学习模型的三级可解释性可以证明输入特征的模型输出是合理的，最后，将通过虚拟交互式行人模拟器中的主题实验来评估所开发的意图预测模型。通过工业合作者共享和会议，并向公众传播行人行为基准数据集，研究结果将纳入工程教育，以推广考虑用户感受、价值观和整体心理状态的设计方法（移情教育设计）。活动将包括在研究者所在大学、自动驾驶研究社区和行业举办的课程。他们将提高对人工智能设计中的偏见、信任和社会智能等以人为本的关键人工智能问题的认识。该奖项反映了 NSF 的法定使命和使命。已被视为值得通过使用基金会的智力优势和更广泛的影响审查标准进行评估来获得支持。

项目成果

Flexible and scalable annotation tool to develop scene understanding datasets

• DOI: 10.1145/3546930.3547499
• 发表时间: 2022-06
• 期刊: Proceedings of the Workshop on Human-In-the-Loop Data Analytics
• 作者: Md. Fazle Elahi Khan;Renran Tian;Xiao Luo

Attention-Based Interrelation Modeling for Explainable Automated Driving

• DOI: 10.1109/tiv.2022.3229682
• 发表时间: 2023-02
• 期刊: IEEE Transactions on Intelligent Vehicles
• 影响因子: 8.2
• 作者: Zhengming Zhang;Renran Tian;Rini Sherony;Joshua E. Domeyer;Zhengming Ding

TrEP: Transformer-Based Evidential Prediction for Pedestrian Intention with Uncertainty

• DOI: 10.1609/aaai.v37i3.25463
• 发表时间: 2023-06
• 作者: Zhengming Zhang;Renran Tian;Zhengming Ding