CHS: Small: DeepCrowd: A Crowd-assisted Deep Learning-based Disaster Scene Assessment System with Active Human-AI Interactions

CHS：小型：DeepCrowd：一种基于人群辅助、基于深度学习的灾难场景评估系统，具有主动人机交互功能

• 批准号: 2008228
• 负责人: Dong Wang
• 金额: $ 49.98万
• 依托单位: University of Notre Dame
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-01-01 至 2021-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008228&HistoricalAwards=false
• 关键词: CHS; Small; DeepCrowd; Crowd; assisted

Recent advances in artificial intelligence (AI) have transformed many important domains of modern life such as transportation, finance, education, healthcare, and entertainment. This project addresses application of AI to disaster scene assessment (DSA). For DSA, artificial intelligence can be used to automatically identify damage severity of impacted areas from imagery reports in the aftermath of a disaster such as earthquake, hurricane, or landslides. A key limitation of AI based techniques is the black-box nature of many contemporary models and the consequent lack of interpretability of the results and failures. This project investigates the problem of troubleshooting, tuning, and eventually improving the black-box AI algorithms by integrating human intelligence with machine intelligence through active crowd-AI interactions. The work complements the prevailing AI solutions that primarily focus on AI model design and training sample collection. The results from this project will open up unprecedented opportunities of fully exploring the wisdom from the crowd in various crowd-assisted AI application domains. This project will also provide opportunities for students in STEM and from underrepresented groups to study the interaction between AI and humans. This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions. In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions. In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）的最新进展改变了现代生活的许多重要领域，例如运输，金融，教育，医疗保健和娱乐。该项目解决了AI在灾难现场评估（DSA）中的应用。对于DSA，可以使用人工智能自动从地震，飓风或山体滑坡等灾难发生后，从图像报告中自动识别受影响区域的损害严重程度。 基于AI的技术的关键局限性是许多当代模型的黑盒本质，因此缺乏结果和失败的解释性。该项目通过通过主动的人群交互将人类智能与机器智能整合到了对黑盒AI算法进行故障排除，调整和最终改善的问题。这项工作补充了主要的AI解决方案，主要集中于AI模型设计和培训样本收集。该项目的结果将为充分探索各种人群辅助AI应用领域的人群的智慧提供前所未有的机会。该项目还将为STEM的学生和代表性不足的群体提供机会研究AI与人类之间的相互作用。该项目开发了一个深脚框架，该框架可用于指导未来的人群应用程序的设计，开发和实施，其中从人群中获得的人类智能与AI深度学习模型紧密整合在一起，以显着提高AI的系统性能 - 只有或只有人类的解决方案。该项目通过AI，机器学习，估算理论和网络人类互动启发的跨学科方法解决了AI和Deepcrowd中的Black-Box挑战和众包平台。特别是，研究包括i）制定人群任务生成计划，以有效查询众包平台以寻求反馈； ii）创建一种新颖的自适应机制，以激励人群及时，准确的反应； iii）设计一种交互式注意力神经网络方案，该方案可以在人群和AI模型之间进行直接相互作用； iv）开发人群和AI集成引擎，该引擎有效地纳入了人群的反馈，以减轻AI的失败情况。由此产生的Deep Crowd框架具有变革性，因为它将生成一系列新的Crowd-ai相互作用模型和技术，以增强系统性能来构建新颖的人群辅助AI应用。开发和实施未来的人群应用程序，其中从人群中获得的人类智能与AI深度学习模型紧密整合在一起，以显着改善系统性能，而不是仅AI或仅由人类的解决方案。该项目通过AI，机器学习，估算理论和网络人类互动启发的跨学科方法解决了AI和Deepcrowd中的Black-Box挑战和众包平台。特别是，研究包括i）制定人群任务生成计划，以有效查询众包平台以寻求反馈； ii）创建一种新颖的自适应机制，以激励人群及时，准确的反应； iii）设计一种交互式注意力神经网络方案，该方案可以在人群和AI模型之间进行直接相互作用； iv）开发人群和AI集成引擎，该引擎有效地纳入了人群的反馈，以减轻AI的失败情况。由此产生的Deep Crowd框架具有变革性，因为它将产生一系列新的人群交互模型和技术，以增强系统性能来构建新颖的人群辅助AI应用程序。该奖项反映了NSF的法定任务，并被视为值得通过评估的支持。利用基金会的知识分子和更广泛的影响审查标准。