CIF: Small: An Algebraic, Convex, and Scalable Framework for Kernel Learning with Activation Functions

CIF：小型：具有激活函数的核学习的代数、凸性和可扩展框架

• 批准号: 2323532
• 负责人: Matthew Peet
• 金额: $ 33.42万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-12-01 至 2026-11-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2323532&HistoricalAwards=false
• 关键词: CIF; Small; Algebraic; Convex; Scalable

Public interest in machine learning has increased significantly in recent years, with application in a diversity of fields, from medical diagnosis to speech recognition to autonomous driving to advertising. The ability to sustain this interest, however, will depend on whether machine learning algorithms continue to advance in terms of both reliability, scalability, and interpretability. As more data becomes available, Will self-driving cars become safer? Will Siri understand you better? Will doctors be able to better understand the causes and treatments for diseases? While neural networks and deep learning have seen widespread adoption in recent years, the algorithms which underly these methods have not changed substantially in over 20 years. This project, therefore, revisits the fundamental mathematics which underly machine learning algorithms – integrating classical results with the popular neural-network based approaches. This mathematical framework is then used to propose new methods for improving the accuracy of machine learning, for increasing the ability to process large data sets, and for allowing the results of machine learning algorithms to be more readily interpreted in terms of measurable physical quantities. To achieve the goals of accuracy, scalability and interpretability, the project poses an algebraic reformulation of the classical problem of learning the kernel. Specifically, for any given kernel algebra, the positive kernels in that algebra and their associated feature maps may be represented by positive matrices – leading to a convex optimization problem whose solution yields an explicit feature map which may be interpreted in terms of measurable physical quantities. Based on this framework, activation functions are used to define kernel algebras which are universal, yet which are dense in the set of all kernels and whose feature maps mimic those of the neural tangent kernel which defines neural networks – leading to improved accuracy of the algorithms. Next, a saddle-point representation and primal-dual approach is used to convert the kernel learning problem to quadratic programming – resulting in more scalable kernel learning algorithms. Finally, a singular value decomposition of the resulting feature map is obtained by solving an associated partial differential equation. This decomposition is used to identify key features in the data and, furthermore, yields reduced algorithms which scale linearly with the number of samples – implying scalability to datasets with tens of thousands of samples.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，公众对机器学习的兴趣已大大增加，从医学诊断到语音识别再到自主驾驶再到广告的各种领域。但是，维持这一兴趣的能力将取决于机器学习算法是否在可靠性，可伸缩性和可解释性方面继续提高。随着越来越多的数据可用，自动驾驶汽车会变得更安全吗？ Siri会更好地了解您吗？医生是否能够更好地了解疾病的原因和治疗方法？尽管近年来神经网络和深度学习已经广泛采用，但这些方法基础的算法在20多年中并没有发生很大变化。因此，该项目重新审视了基础机器学习算法的基本数学 - 将经典结果与流行的基于神经网络的方法相结合。然后，该数学框架被用来提出新的方法，以提高机器学习的准确性，提高处理大数据集的能力以及允许用可测量的物理量来更容易解释机器学习算法的结果。为了实现准确性，可伸缩性和解释性的目标，该项目对学习内核的经典问题提出了代数改革。具体而言，对于任何给定的内核代数，该代数及其相关特征图中的正核可以由积极物质代表 - 导致凸优化问题，其解决方案会产生明确的特征图，该特征图可以用可测量的物理量来解释。基于此框架，激活函数用于定义通用的内核代数，但在所有内核中都很致密，并且它们的特征映射模拟了神经元切线核的特征，该神经元切线核的特征是定义神经网络的，从而提高了算法的准确性。接下来，使用鞍点表示和原始的双重方法将内核学习问题转换为二次编程，从而产生了更可扩展的内核学习算法。最后，通过求解相关的部分微分方程来获得所得特征图的单数值分解。该分解用于识别数据中的关键特征，此外，算法减少了与样本数量线性缩小的算法 - 这意味着具有数万个样本的可扩展性对数据集进行了可扩展性。该奖项反映了NSF的法规任务，并认为通过基金会的知识绩效和广泛的cribia criperia criperia criperia the Insportaucation the Pocies taundia the Iss Issportiation。