Optimization-based Implicit Deep Learning, Theory and Applications

基于优化的隐式深度学习、理论与应用

• 批准号: 2309810
• 负责人: Samy Wu Fung
• 金额: $ 29.5万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-15 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2309810&HistoricalAwards=false
• 关键词: Optimization; based; Implicit; Deep; Learning

The past decade has seen remarkable success in deep learning. However, a significant challenge in today's era is to ensure interpretability and reliability in these models. In various applications, deep neural networks (DNNs) need to provide guarantees on their outputs, such as maintaining a self-driving car within its lane. On the other hand, many of these tasks can be formulated as optimization problems, where optimization algorithms offer interpretable and reliable solutions. Unfortunately, these models do not leverage data and thus fall short of state-of-the-art deep learning models. This research will address enhancing interpretability and reliability in deep learning methods and improve public safety when such learning methods are applied. In addition, the project will provide valuable educational opportunities for students involved. Participants will gain knowledge in inverse problems, optimization, and machine learning, which are transferable skills applicable in academia, government, and industry. The project aims to develop a framework that combines the interpretability and reliability of optimization algorithms with the design and training of DNNs. The primary focus is on implicit networks, a type of DNNs that determines their outputs implicitly through fixed point or optimality conditions, rather than a fixed number of computations like traditional DNNs with a set number of layers. This integration of optimization algorithms into implicit networks is referred to as implicit learning-to-optimize (L2O) networks. Implicit L2O networks have the potential to overcome the limitations of traditional DNNs, including their lack of reliability and interpretability. However, training and designing implicit L2O models present additional challenges that hinder their widespread adoption. To address these challenges, the research aims to develop a universal implicit L2O framework.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

过去十年在深度学习中取得了巨大的成功。但是，在当今时代的一个重大挑战是确保这些模型中的可解释性和可靠性。在各种应用中，深层神经网络（DNN）需要为其产出提供保证，例如在其车道内保持自动驾驶汽车。另一方面，其中许多任务可以作为优化问题提出，优化算法提供了可解释且可靠的解决方案。不幸的是，这些模型不利用数据，因此缺乏最新的深度学习模型。这项研究将解决深度学习方法中的可解释性和可靠性，并在应用这种学习方法时提高公共安全。此外，该项目将为参与学生提供宝贵的教育机会。参与者将获得相反问题，优化和机器学习的知识，这些技能适用于学术界，政府和行业。该项目旨在开发一个将优化算法的可解释性和可靠性与DNN的设计和培训相结合的框架。主要的重点是隐式网络，一种DNN的类型，它通过固定点或最佳条件暗中确定其输出，而不是固定数量的计算，例如传统的DNN，具有一定的层。将优化算法集成到隐式网络中的这种集成被称为隐式学习到优势（L2O）网络。隐式L2O网络有可能克服传统DNN的局限性，包括缺乏可靠性和解释性。但是，培训和设计隐式L2O模型提出了其他挑战，从而阻碍了他们广泛采用。为了应对这些挑战，该研究旨在开发一个普遍的隐式L2O框架。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准来评估的。