Collaborative Research: RI: Small: Advancing Theory and Practice of Trustworthy Machine Learning via Bi-Level Optimization

合作研究：RI：小型：通过双层优化推进可信机器学习的理论和实践

• 批准号: 2207052
• 负责人: Sijia Liu
• 金额: $ 30万
• 依托单位: Michigan State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2207052&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Advancing

Deep learning (DL) has achieved remarkable success owing to its superior prediction ability, with a wide range of applications in computer vision and natural language processing. Yet, one of its critical shortcomings is the lack of trustworthiness. That is, they are often overcooked during training such that (1) the learned model is highly vulnerable to small input perturbations at the testing time (namely, lack of robustness); And (2) biased artifacts embedded in the training data can be memorized and then passed on to the decision making process (namely, lack of fairness). To address these issues, this project attempts to develop a new family of trustworthy learning algorithms with algorithmic generality, theoretical soundness, and scalability to large-scale datasets and models. The outcome of this project could create a new optimization foundation of trustworthy DL that can not only unit robustness and fairness into one coherent learning paradigm but also expand the applicability of DL to a series of high-stakes applications such as autonomous driving and cybersecurity. Interdisciplinary training in computer science, applied mathematics, and engineering will be provided to all-level students, especially for students from underrepresented groups. The main technical aim of this project is to advance the theoretical understanding and practical implementations of trustworthy DL through the lens of bi-level optimization (BLO), namely, hierarchical learning involving two nested optimization tasks. The research plan consists of three thrusts. The first thrust develops a new BLO-oriented robust learning framework including defenses against adversarial instances and distribution shifts. The developed technique is also applied to building a full-stack (from train time to test time) robustness evaluation pipeline. The second thrust expands the first one and develops BLO algorithms to co-improve robustness and fairness in two practical scenarios, learning without sensitive attribute annotation, and learning with scarce training data and model information. The third thrust focuses on developing scalable and theoretically-grounded computational methods for BLO so as to achieve a high-accuracy, high-resilience, and high-throughput trustworthy learning paradigm. The project will result in the dissemination of shared toolbox and benchmarks to the broader optimization and machine learning communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习（DL）凭借其卓越的预测能力取得了显着的成功，在计算机视觉和自然语言处理领域有着广泛的应用。然而，其关键缺点之一是缺乏可信度。也就是说，它们在训练过程中经常被过度训练，使得（1）学习的模型在测试时非常容易受到小输入扰动的影响（即缺乏鲁棒性）； (2) 训练数据中嵌入的有偏差的工件可以被记忆，然后传递到决策过程（即缺乏公平性）。为了解决这些问题，该项目试图开发一系列新的值得信赖的学习算法，这些算法具有算法通用性、理论可靠性以及对大规模数据集和模型的可扩展性。该项目的成果可以为值得信赖的深度学习创建一个新的优化基础，不仅可以将鲁棒性和公平性整合到一个连贯的学习范式中，还可以将深度学习的适用性扩展到自动驾驶和网络安全等一系列高风险应用。将为所有级别的学生，特别是来自代表性不足群体的学生提供计算机科学、应用数学和工程学的跨学科培训。该项目的主要技术目标是通过双层优化（BLO）（即涉及两个嵌套优化任务的分层学习）的视角，推进可信赖深度学习的理论理解和实际实现。该研究计划由三个重点组成。第一个重点是开发一个新的面向 BLO 的稳健学习框架，包括防御对抗性实例和分布变化。开发的技术还应用于构建全栈（从训练时间到测试时间）的鲁棒性评估管道。第二个推力扩展了第一个推力，开发了 BLO 算法，以在两种实际场景中共同提高鲁棒性和公平性：无需敏感属性注释的学习以及稀缺训练数据和模型信息的学习。第三个重点是为 BLO 开发可扩展且有理论基础的计算方法，以实现高精度、高弹性和高吞吐量的可信学习范式。该项目将导致向更广泛的优化和机器学习社区传播共享工具箱和基准。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。