Elements: CausalBench: A Cyberinfrastructure for Causal-Learning Benchmarking for Efficacy, Reproducibility, and Scientific Collaboration

要素：CausalBench：用于因果学习基准测试的网络基础设施，以实现有效性、可重复性和科学协作

• 批准号: 2311716
• 负责人: Kasim Candan
• 金额: $ 59.99万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311716&HistoricalAwards=false
• 关键词: Elements; CausalBench; Cyberinfrastructure; Causal; Learning

While we are witnessing the exceptional success of artificial intelligence (AI) and machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of the current approaches: they are not causally grounded. While being relatively recent, causal learning aims to go far beyond conventional machine learning and is emerging as a vibrant field with new opportunities and challenges. Yet, advances in this field are hampered due to the lack of cyber-infrastructure platforms, with unified benchmarks data sets, algorithms, metrics, and evaluation service interfaces for causal learning. Reproducible science is possible only when the outcomes can be quantified and compared to other approaches and lack of reproducibility results in serious concerns on validity of published research. This can only be achieved through open platforms for data, algorithm, and model exchange and evaluation. Therefore, CausalBench, a transparent, fair, and easy-to-use evaluation platform, provides the key functionalities necessary to establish trust in causal learning’s innovation, collaboration, and critical applications, including public health and sustainability.CausalBench is a novel cyberinfrastructure of benchmarking data, algorithms, models, and metrics for causal learning, impacting the needs of a broad of scientific and engineering disciplines and sustain discovery across all fields. The cyberinfrastructure enables the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and promotes scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench includes (1) an “ontology” for benchmarking to standardize the evaluation methodology, improve transparency, and promote collaboration to efficiently advance causal learning, (2) standard and convenient mechanisms for the community to contribute data and models such that disparate datasets can be integrated in a standard way, and (3) integrated evaluation standards that can help assess of algorithms for novel problems in the emerging field of causal learning with observational data. The project trains PhD students in the area of causal discovery and causally aware data management challenges via integrative, cross-disciplinary approaches and prepares future researchers with skills in data intensive AI and machine learning systems. The project further provides an excellent context for master’s, undergraduate, and K-12 students to be aware of AI and machine learning and their potential impacts on urgent societal challenges including public health, and sustainability.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Information and Intelligent Systems within the Computer and Information Science and Engineering Directorate.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

虽然我们见证了人工智能 (AI) 和机器学习 (ML) 技术在许多应用中取得的巨大成功，但用户开始注意到当前方法的一个关键缺点：它们不是基于因果关系的，而是相对较新的因果学习。旨在远远超越传统的机器学习，并正在成为一个充满新机遇和挑战的充满活力的领域，然而，由于缺乏网络基础设施平台以及统一的基准数据集、算法、指标和数据，该领域的进步受到阻碍。用于因果学习的评估服务接口。只有当结果可以量化并与其他方法进行比较时，可重复的科学才可能实现，而缺乏可重复性会导致对已发表研究的有效性的严重担忧，这只能通过数据、算法和模型交换和评估的开放平台来实现。 CausalBench 是一个透明、公平且易于使用的评估平台，提供了建立对因果学习创新、协作和关键应用（包括公共卫生和可持续性）的信任所需的关键功能。CausalBench 是一个新颖的用于因果学习的基准数据、算法、模型和指标的网络基础设施，影响广泛的科学和工程学科的需求，并维持所有领域的发现。网络基础设施通过促进新颖算法的科学合作，促进因果学习研究的进步。 CausalBench 包括 (1) 用于基准测试的“本体论”，以标准化评估方法、提高透明度并促进因果学习研究的科学客观性、可重复性、公平性和偏见意识。协作以有效推进因果学习，（2）社区贡献数据和模型的标准和便捷的机制，以便可以以标准方式集成不同的数据集，以及（3）集成的评估标准，可以帮助评估新问题的算法该项目通过综合、跨学科的方法，在因果发现和因果意识数据管理挑战领域培训博士生，并为未来的研究人员提供数据密集型人工智能和机器学习方面的技能。该项目进一步为系统提供了良好的环境。旨在帮助硕士、本科生和 K-12 学生了解人工智能和机器学习及其对公共卫生和可持续发展等紧迫社会挑战的潜在影响。该奖项由先进网络基础设施办公室颁发，并得到信息与智能部门的共同支持计算机和信息科学与工程理事会内的系统。该奖项反映了 NSF 的法定使命，并且通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。