QMIA: Quantifying and Mitigating Bias affecting and induced by AI in Medicine

QMIA：量化和减轻人工智能在医学中影响和诱发的偏差

• 批准号: MR/X030075/1
• 负责人: Honghan Wu
• 金额: $ 82.72万
• 依托单位: University College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=MR%2FX030075%2F1
• 关键词: QMIA; Quantifying; Mitigating; Bias; affecting

Artificial Intelligence (AI) has demonstrated exciting potential in improving healthcare. However, these technologies come with a big caveat. They do not work effectively for minority groups. A recent study published in Science shows a widely used AI tool in the US concludes Black patients are healthier than equally sick Whites. Using this tool, a health system would favour White people when allocating resources, such as hospital beds. AI models like this would do more harm than good for health equity. Such inequality goes way beyond racial groups, affecting people with different gender, age and socioeconomics background. Such AI induced bias might come from healthcare data, which significantly lacks data on minorities and embeds decades of health care disparities among different groups of people. The COVID-19 pandemic highlighted this issue, with UK minority groups disproportionately affected by higher infection rates and worse outcomes. Bias may also arise in the design and development of AI tools, where inequalities can be built into the decisions they make, including how to characterise patients and what to predict. For example, the above-mentioned AI tool in the US uses health costs as a proxy for health needs, making its predictions reflect economic inequality as much as care requirements, further perpetuating racial disparities. However, currently, AI models in medicine are still only measured by accuracy, leaving their impact on inequalities untested. Current AI audit tools are not fit for purpose as they do not detect and quantity bias based on actual health needs. Largely absent are effective tools devised particularly for healthcare for evaluating and mitigating AI induced inequalities. This project aims to develop a set of tools for optimising health datasets and supporting AI development in ensuring equity. Central to the solution is a novel measurement tool for quantifying health inequalities: deterioration-allocation area under curve. This framework assess the fairness by checking whether the AI allocate the same level of resources for people with the same health needs across different groups. We will use three representative health datasets: (1) CVD-COVID-UK, containing person-level data of 57 million people in England; (2) SCI-Diabetes, a diabetes research cohort containing everyone with diabetes in Scotland; (3) UCLH dataset, routine secondary care data from University College London Hospitals NHS Foundation Trust. COVID-19 and Type 2 diabetes will be used as exemplar diseases for investigations. Specifically, this project will conduct three lines of work: 1. Analyse the embedded racial bias in all three heath datasets so AI developers can make informed decisions and selections on how to characterise patients and what to predict;2. Systematically review and analyse risk prediction models, particularly those widely used in clinical settings, for COVID-19 and type 2 diabetes;3. Develop a novel method called multi-objective ensemble to bring insights from complementary datasets (avoiding actual data transfer) for mitigating inequality caused by too little data for certain groups. We will work closely with patients and members of the public to help focus and interpret our research, and to help publicise our findings. We will collaborate with other research teams to share learnings and methods, and with the NHS and government to ensure this research turns into practical improvements in health equity.

人工智能（AI）在改善医疗保健方面表现出了令人兴奋的潜力。但是，这些技术引起了很大的警告。他们对少数群体没有有效的工作。科学上发表的最新研究表明，美国广泛使用的AI工具得出结论，黑人患者比同样病的白人更健康。使用此工具，卫生系统在分配资源（例如医院病床）时会喜欢白人。这样的AI模型对健康公平的弊大于利。这种不平等超越了种族群体，影响具有不同性别，年龄和社会经济背景的人们。这种AI引起的偏见可能来自医疗保健数据，这些数据大大缺乏有关少数民族的数据，并且嵌入了不同人群之间数十年的医疗保健差异。 COVID-19大流行强调了这个问题，英国少数群体受到较高的感染率和较差的结果的影响不成比例。 AI工具的设计和开发中也可能出现偏见，在这些工具的设计和开发中，他们做出的决策可能会内置不平等，包括如何表征患者和预测什么。例如，美国上述的AI工具将健康成本作为健康需求的代理，使其预测反映了经济不平等，以及护理要求，进一步延续了种族差异。但是，目前，医学中的AI模型仍然仅通过准确性来衡量，从而对未经测试的不平等影响产生了影响。当前的AI审核工具不适合目的，因为它们无法根据实际的健康需求检测和数量偏差。在很大程度上缺乏有效的工具，尤其是用于评估和减轻AI引起的不平等现象的医疗保健。该项目旨在开发一组工具，以优化健康数据集并支持AI开发以确保股权。该解决方案的中心是用于量化健康不平等的新型测量工具：曲线下的分配分配区域。该框架通过检查AI是否为不同群体具有相同健康需求的人分配相同水平的资源来评估公平性。我们将使用三个代表性的健康数据集：（1）CVD-COVID-UK，其中包含英格兰5700万人的人级数据； （2）科幻糖尿病，糖尿病研究队列，其中包含苏格兰糖尿病的每个人； （3）UCLH数据集，伦敦大学医院NHS基金会信托基金会的常规二级护理数据。 COVID-19和2型糖尿病将用作研究的典范疾病。具体而言，该项目将进行三种工作：1。分析所有三个Heath数据集中嵌入的种族偏见，以便AI开发人员可以就如何表征患者和预测的内容做出明智的决定和选择； 2。系统地审查和分析风险预测模型，尤其是在临床环境中广泛用于Covid-19和2型糖尿病的风险预测模型； 3。开发一种称为多目标集合的新颖方法，以从补充数据集中带来洞察力（避免实际数据传输），以减轻某些组的数据太少而导致的不平等。我们将与患者和公众紧密合作，以帮助专注和解释我们的研究，并帮助宣传我们的发现。我们将与其他研究团队合作，分享学习和方法，并与NHS和政府合作，以确保这项研究变成了健康公平的实际改善。