EAGER: Knowledge-guided neurosymbolic AI with guardrails for safe virtual health assistants

EAGER：知识引导的神经符号人工智能，带有安全虚拟健康助手的护栏

• 批准号: 2335967
• 负责人: Amit Sheth
• 金额: $ 20万
• 依托单位: University of South Carolina at Columbia
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2335967&HistoricalAwards=false
• 关键词: EAGER; Knowledge; guided; neurosymbolic; AI

This project addresses the limitations of generative artificial intelligence (AI) systems, particularly in the context of virtual assistants used to support healthcare (VHAs). While VHAs show potential for empowering patients and addressing clinical expertise shortages, concerns about safety and accessibility arise due to inaccuracies in their outputs and their lack of adherence to the relevant standards of care. To mitigate these concerns, the project proposes an innovative approach for integrating clinical protocols and practice guidelines within AI systems. This approach will enable the development of safety constrained VHAs that support clinicians and ensure safe interactions with patients. Additionally, the approach facilitates the provision of clinician-friendly explanations, fostering improved collaboration between humans and AI in healthcare. By addressing significant current concerns surrounding the safety of generative AI, the research will promote user confidence and adoption in safety-critical domains requiring human-AI collaboration. The research's success can have implications beyond healthcare, such as autonomous vehicles incorporating traffic rules or manufacturing processes ensuring safe operations and maintenance compliance. Furthermore, the project aligns with efforts to promote inclusivity in computing, workforce development, and education. Example initiatives include annual AI summer camp for school students from underrepresented backgrounds, and engagement with high school, undergraduate and graduate students through internships and workforce development modules relevant to interdisciplinary AI careers.The main innovation of this research lies in leveraging Knowledge Graphs (KGs) to construct guardrails that help ensure the safety of AI systems. In collaboration with clinical experts, a KG enriched with both declarative (e.g., medical terminology and definitions) and procedural or process knowledge (e.g., diagnostic criteria and clinical practice guidelines) will be employed to guide neural processing architectures, resulting in the development of VHAs inherently constrained to be safe. Furthermore, the same proposed methods will also equip VHAs with the ability to generate end user (e.g., clinician) friendly explanations making the system verifiable. The project's two important outcomes will be to (a) effectively apply medical guidelines from the KG to uphold high safety standards in a clinical setting, and (b) generate explanations that are easily comprehensible to end users using the terms, concepts, and guidelines relevant to end-user verification and decision-making. The core techniques proposed in this project will advance the state-of-the-art in neurosymbolic AI toward facilitating robust (verifiable and safety-constrained) collaboration between humans and AI. These advances have the potential for transferability to other domains with safety-critical applications, thus contributing to the broader field of AI research and its wider adoption.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目解决了生成人工智能（AI）系统的局限性，尤其是在用于支持医疗保健（VHAS）的虚拟助理的背景下。尽管VHA显示出赋予患者能力并解决临床专业知识短缺的潜力，但由于其产出不准确以及他们缺乏遵守相关护理标准而引起的安全性和可及性的担忧。为了减轻这些问题，该项目提出了一种创新的方法，将临床方案和实践指南整合在AI系统中。这种方法将能够开发安全约束的VHA，以支持临床医生并确保与患者的安全互动。此外，该方法有助于提供对临床医生友好的解释，从而促进了人类与AI医疗保健方面的合作。通过解决围绕生成AI安全性的当前问题，该研究将促进用户信心和对需要人类协作的安全至关重要领域的采用。这项研究的成功可能会产生超出医疗保健的影响，例如合并交通规则或制造过程的自动驾驶汽车，以确保安全操作和维护合规性。此外，该项目与促进计算，劳动力发展和教育方面的包容性努力保持一致。示例举措包括一年一度的来自代表性不足背景的学校学生的年度AI夏令营，以及通过实习和与跨学科AI职业相关的高中，本科和研究生的互动，这项研究的主要创新在于利用知识图（KGS），以帮助确保构建护卫队的载体。与临床专家合作，将使用宣言性（例如医学术语和定义）以及程序或过程知识（例如，诊断标准和临床实践指南）的公园来指导神经处理架构，从而导致VHA固有地限制的VHA的发展。此外，相同提出的方法还将为VHA提供能力生成最终用户（例如临床医生）友好解释，从而使系统可验证。该项目的两个重要结果将是（a）在临床环境中有效地将医疗指南应用于高安全标准，并且（b）生成解释，这些解释易于使用与最终用户验证和决策的术语，概念和指南相关的最终用户易于理解。该项目中提出的核心技术将推动神经合格AI的最新技术，以促进人类与AI之间的强大（可验证和安全约束）合作。这些进步有可能通过安全至关重要的应用转移到其他领域，从而有助于更广泛的AI研究领域及其更广泛的采用领域。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响审查标准通过评估来获得支持的。