Optimization-based Implicit Deep Learning, Theory and Applications

基于优化的隐式深度学习、理论与应用

• 批准号: 2309810
• 负责人: Samy Wu Fung
• 金额: $ 29.5万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-15 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2309810&HistoricalAwards=false
• 关键词: Optimization; based; Implicit; Deep; Learning

The past decade has seen remarkable success in deep learning. However, a significant challenge in today's era is to ensure interpretability and reliability in these models. In various applications, deep neural networks (DNNs) need to provide guarantees on their outputs, such as maintaining a self-driving car within its lane. On the other hand, many of these tasks can be formulated as optimization problems, where optimization algorithms offer interpretable and reliable solutions. Unfortunately, these models do not leverage data and thus fall short of state-of-the-art deep learning models. This research will address enhancing interpretability and reliability in deep learning methods and improve public safety when such learning methods are applied. In addition, the project will provide valuable educational opportunities for students involved. Participants will gain knowledge in inverse problems, optimization, and machine learning, which are transferable skills applicable in academia, government, and industry. The project aims to develop a framework that combines the interpretability and reliability of optimization algorithms with the design and training of DNNs. The primary focus is on implicit networks, a type of DNNs that determines their outputs implicitly through fixed point or optimality conditions, rather than a fixed number of computations like traditional DNNs with a set number of layers. This integration of optimization algorithms into implicit networks is referred to as implicit learning-to-optimize (L2O) networks. Implicit L2O networks have the potential to overcome the limitations of traditional DNNs, including their lack of reliability and interpretability. However, training and designing implicit L2O models present additional challenges that hinder their widespread adoption. To address these challenges, the research aims to develop a universal implicit L2O framework.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

过去十年，深度学习取得了巨大的成功。然而，当今时代的一个重大挑战是确保这些模型的可解释性和可靠性。在各种应用中，深度神经网络 (DNN) 需要为其输出提供保证，例如将自动驾驶汽车保持在其车道内。另一方面，其中许多任务可以表述为优化问题，其中优化算法提供可解释且可靠的解决方案。不幸的是，这些模型没有利用数据，因此达不到最先进的深度学习模型。这项研究将致力于增强深度学习方法的可解释性和可靠性，并在应用此类学习方法时改善公共安全。此外，该项目将为参与的学生提供宝贵的教育机会。参与者将获得反问题、优化和机器学习方面的知识，这些都是适用于学术界、政府和工业界的可转移技能。该项目旨在开发一个框架，将优化算法的可解释性和可靠性与 DNN 的设计和训练相结合。主要关注点是隐式网络，这是一种通过定点或最优条件隐式确定其输出的 DNN，而不是像具有一定层数的传统 DNN 那样进行固定数量的计算。这种将优化算法集成到隐式网络中的方式称为隐式学习优化 (L2O) 网络。隐式 L2O 网络有潜力克服传统 DNN 的局限性，包括缺乏可靠性和可解释性。然而，训练和设计隐式 L2O 模型带来了额外的挑战，阻碍了其广泛采用。为了应对这些挑战，该研究旨在开发一个通用的隐式 L2O 框架。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。