RAPID/Collaborative Research: Human-AI Teaming for Big Data Analytics to Enhance Response to the COVID-19 Pandemic

快速/协作研究：人类与人工智能合作进行大数据分析以增强对 COVID-19 大流行的响应

• 批准号: 2029692
• 负责人: Keri Stephens
• 金额: $ 2.03万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-15 至 2021-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2029692&HistoricalAwards=false
• 关键词: RAPID; Collaborative; Research; Human; AI

Social media data can provide important clues and local knowledge that can help emergency managers and responders better comprehend and capture the evolving nature of many disasters. Yet humans alone cannot grasp the vast data generated by social media, so computers are used to assist. Very little is currently known about how to leverage the skills of humans and machines when they work together (human-machine teaming) to identify meaningful patterns in social media data. Therefore, the fundamental issues this Rapid Response Research (RAPID) project seeks to address are 1) understanding the process of real-time decisions that human digital volunteers make when they rapidly convert social media data into structured codes the machine (Artificial Intelligence algorithms) can understand, and 2) using this knowledge to improve human-machine teaming. This project advances the field by revealing the unique abilities that both humans and machines bring when working together to comprehend social media patterns during an evolving disaster. It supports education and diversity by providing research experiences to diverse students, as well as generating data useful for interdisciplinary courses teaching teamwork, social media analysis, and human-machine teaming. Finally, the findings can help emergency managers better train their volunteers who comb through social media using their understanding of the local knowledge and built environment to help machines see new patterns in data. Hence, this project supports NSF's mission to promote the progress of science and to advance the nation's health, prosperity, and welfare by articulating the unique value that both humans and computers bring that can lead to better decisions during disasters. The goal of this research is to better understand the real-time decisions that human annotators make under different environmental constraints, and how those contribute to the learning of Artificial Intelligence (AI) models. Under time constraints and information overload, human decision-making capabilities are limited; yet, humans still have a unique ability to understand the contextual references to the structures in the built environment that machines cannot recognize. For example, the meaning of the tweet, “Memorial is overloaded,” -- which means the hospital, called Memorial, is out of beds for patients —- can be lost on AI systems that lack the knowledge of the built environment. This example demonstrates the value that humans in the loop offer in a human-AI teaming context. This research focuses on capturing the ephemeral data from a variety of social media sources and our two research thrusts include: 1) online observations of Community Emergency Response Team (CERT) volunteers and a manager (a collaborator on this project) using think-aloud and cognitive interviewing strategies to reveal the real-time mental models used to make coding decisions for annotation tasks; and 2) an empirical analysis of different sampling algorithms for active (machine) learning paradigms to develop a typology of machine errors under diverse contexts that affect the quality of human decision making for annotation. This research will generate design guidelines that bridge the gap between the mechanisms used for real-time data processing with AI models and the understanding of context contributed by a human user teaming with the AI models. Using theories of human decision-making combined with knowledge of how AI functions, this project provides a real-time, mid-disaster examination of 1) how humans understand, process, and interpret social media messages, and 2) how to refine AI algorithms to optimize active learning paradigm. This understanding will provide a theoretical framework enabling future research to develop protocols to optimize human-AI teaming by using concepts such as motivation and information theory. This work can help emergency managers conduct better training of their CERT volunteers and other annotators and provide clearer guidelines for how to communicate the unique value that humans bring to the annotation process for AI systems. Both our protocols and developed understanding of how humans interact with AI systems will be helpful for global health organizations, local and state-level disaster decision-makers, as well as provide direction for the vast CERT network in the United States.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Social Media Data Can Provide Important Clues and Local Knowledge That CAN HELP EMERGENCY MANAGERS AND RESPONDERS BETTER CAPOLEND CAPTURE THE TOLE THE TOLE TOLE TOLEE THELE TOLEE T some G Nature of Many Disasters. Yet Humans Alone Cannot Grasp The Vast Data Generated by Social Media, SO计算机被用来协助人类的技能，并共同努力（人机团队）社交媒体数据。 - 当时的决定，然后然后将媒介媒体数据纳入机器（人工智能算法），以及2）使用这些知识来改善人类机器小镇，以陶醉于人类和机器时。在ANN G灾难中共同理解社交媒体模式。建立的环境可帮助机器看到数据中的新模式这项研究是为了更好地做出的实时决定，即人类注释在时间限制和信息过载下产生不同的环境限制（AI）模型，人类的决策能力仍然有限。对于机器无法识别的构建环境这项研究重点介绍来自各种社交媒体来源的短暂数据，我们的两个推力包括：1）使用Think-Aloud和认知访谈策略对委员会出现团队（CERT）的在线观察用于对注释任务做出编码决策p机器错误的类型上下文是人类决策制定的质量。 AI模型使用人类决策理论以及对AI功能的知识，提供了实时的，中间的检查1）Wumans了解，过程和解释社交媒体信息HMS优化积极的学习范式。其他注释者，并为如何传达人类为AI系统的注释过程传达的独特价值提供了更清晰的准则。国家。该奖项反映了NSF'Shas值得一提的是使用Toundation IT和更广泛的影响审查标准进行支持评估。