Collaborative Research: Algebraic Framework of Compositional Functions for New Structure, Training, and Explainability of Deep Learning

合作研究：深度学习新结构、训练和可解释性的组合函数代数框架

• 批准号: 2134235
• 负责人: Qi Gong
• 金额: $ 45.85万
• 依托单位: University of California-Santa Cruz
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2134235&HistoricalAwards=false
• 关键词: Collaborative; Research; Algebraic; Framework; Compositional

Deep learning is a method of machine learning inspired by the human brain. Data is fed to a multi-layered (deep) network of trainable 'neurons' and the network is then trained to model complex relations and processes. Deep learning has had impressive success in applications such as image recognition and natural language processing. And yet, there are few theoretical guarantees to the method, to provide assurances in regards to performance features such as error and to explain its success. This is an impediment to broader application of deep learning, as many potential applications require guarantees for safety, reliability, and accuracy. This project pursues a solid mathematical foundation for a better understanding of the explainability of deep learning, to enable more efficient neural network design and training algorithms that benefit a wide range of applications. The project also includes a significant educational component that is designed to foster interdisciplinary education by engaging undergraduate and graduate students from the investigators' departments (Applied Mathematics, Electrical Engineering, Computer Science, Mathematics) in the proposed multidisciplinary research. The project includes plans to promote diversity, equity and inclusion in STEM education at the University of California Santa Cruz and the University of Texas at San Antonio, which are both Hispanic Serving Institutions.The overarching goal of this project is to develop a unified algebraic framework and approximation theory for deep neural networks so that the framework is applicable to a wide spectrum of problems including regression, solving differential equations, designing optimal feedback control, and computer vision. The proposed research is motivated by the fact that most complicated and high dimensional input-output relations in real-world applications can be represented as compositions of simple low-dimensional functions. Thus, compositional functions, including deep neural networks, serve as a natural way to describe complex high dimensional functions. Representing compositional functions as layered acyclic graphs, the project will explore the compositional features of the problems to be solved by machine learning; study the error propagation in layered acyclic graphs; and investigate the interconnection between the compositional features and the fundamental issues of machine learning, such as the error bounds in universal approximation, deep neural network design and training, and validation and explainability. The algebraic framework, approximation theory, and computational algorithms to be developed in this research project should advance the design, training, and mathematical foundations of deep learning. They seek also to be directly applicable to a wide spectrum of applications including feedback control design and computer vision, which are included in this project, as well as other important machine learning applications, such as regression and data-driven modeling of dynamical systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习是一种受人脑启发的机器学习方法。数据被馈送到可训练的“神经元”的多层（深）网络，然后对网络进行训练以建模复杂的关系和过程。深度学习在图像识别和自然语言处理等应用中取得了令人印象深刻的成功。然而，该方法几乎没有理论保证，可以提供有关绩效特征（例如错误和解释成功）的保证。这是对深度学习的更广泛应用的障碍，因为许多潜在的应用需要保证安全，可靠性和准确性。该项目旨在更好地理解深度学习的解释性，以实现更有效的神经网络设计和培训算法，从而有利于广泛的应用。该项目还包括一个重要的教育组成部分，旨在通过在拟议的多学科研究中参与研究人员部门的本科生和研究生（应用数学，电气工程，计算机科学，数学）来促进跨学科教育。该项目包括促进加州大学圣克鲁斯分校和得克萨斯大学圣安东尼奥分校在STEM教育中促进多样性，公平和包容性的计划，这既是西班牙裔服务机构。该项目的总体目标是开发一个统一的代数框架和近似近似的框架，包括框架的范围，以便框架进行典范，以使框架适用于范围，以使框架适用于范围，以使框架适用于范围，以使框架适用于范围，以使框架适用于范围，以使框架适用于范围。控制和计算机视觉。拟议的研究是由以下事实激发的：在现实世界应用中，最复杂和高维输入关系可以表示为简单的低维函数的组成。因此，包括深神经网络在内的组成函数是描述复杂高维功能的自然方式。该项目将组成函数表示为分层的无环图，将探讨通过机器学习解决的问题的组成特征；研究分层无环图中的误差传播；并研究组成特征与机器学习的基本问题之间的互连，例如通用近似中的误差界限，深度神经网络设计和培训以及验证和解释性。该研究项目中要开发的代数框架，近似理论和计算算法应推进深度学习的设计，培训和数学基础。他们还寻求直接适用于广泛的应用程序，包括反馈控制设计和计​​算机视觉，包括该项目中的其他重要机器学习应用，例如对动态系统的回归和数据驱动的建模。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识优点和广泛的cribitia crietia的评估来通过评估来进行评估。

项目成果

Compositional Features and Neural Network Complexity in Deep Learning

深度学习中的组成特征和神经网络复杂性

• 发表时间: 2022
• 期刊: 25th International Symposium on Mathematical Theory of Networks and Systems (MTNS
• 作者: Gong, Qi;Kang, Wei
• 通讯作者: Kang, Wei

An Actor Critic Method for Free Terminal Time Optimal Control

自由终端时间最优控制的Actor批评方法

• 发表时间: 2023
• 期刊: The 12th IFAC Symposium on Nonlinear Control Systems (NOLCOS
• 作者: Burton, Evan;Nakamura-Zimmerer, Tenavi;Gong, Qi;Kang, Wei
• 通讯作者: Kang, Wei

The Observability in Unobservable Systems

不可观测系统的可观测性

• DOI: 10.1109/icca54724.2022.9831888
• 发表时间: 2022
• 期刊: Italy
• 作者: Kang, Wei;Xu, Liang;Zhou, Hong
• 通讯作者: Zhou, Hong

Approximation of compositional functions with ReLU neural networks

使用 ReLU 神经网络逼近复合函数

• 发表时间: 2023
• 期刊: Systems control letters
• 作者: Gong, Q.;Kang, W.;Fahroo, F.
• 通讯作者: Fahroo, F.

Neural Network Optimal Feedback Control With Guaranteed Local Stability

保证局部稳定性的神经网络最优反馈控制

• DOI: 10.1109/ojcsys.2022.3205863
• 发表时间: 2022
• 期刊: IEEE Open Journal of Control Systems
• 作者: Nakamura-Zimmerer, Tenavi;Gong, Qi;Kang, Wei
• 通讯作者: Kang, Wei