大规模核方法积分算子谱分析的模型选择方法

Whether the inductive bias of the machine learning algorithm maps the problem under consideration directly affects the learning performance of the machine learning algorithm. Model selection of large-scale kernel methods is the process to determine the inductive bias of large-scale kernel methods, which is the bottleneck and key to the theoretical research and application of large-scale kernel methods. In the existing approaches of model selection, the explicit description of the feature space induced by the kernel function is ignored, therefore the model selection of kernel methods lacks of interpretation; Current model selection criteria and algorithms are based on the accurate computation upon the full data set, which suffer poor scalability; In most of existing large-scale kernel methods, a model is selected based on the priori knowledge or the limitation of computational resources, which lacks of theoretical basis. To address these issues, we propose an approach to the model selection of large-scale kernel methods based on the spectral analysis of integral operator. First, we explicitly construct the integral operator space and present an explicit description of the feature space, which can enhance the interpretability of the model. Then, we study the spectral generalization bound from the viewpoints of stability and model complexity, and build the spectral generalization theory of model selection for large-scale kernel methods. Finally, we design spectral model selection criteria and algorithms with interpretation,theoretical guarantee and high computational efficiency. The proposed model selection approach provides an effective way to model selection of large-scale kernel methods, and develops the learning theory and method of large-scale kernel methods.

机器学习算法的归纳偏置是否与问题本身匹配，直接决定学习算法的性能。大规模核方法模型选择是确定大规模核学习算法归纳偏置的过程，是大规模核方法理论研究和实际应用的瓶颈和关键。现有核方法模型选择方法缺乏对核函数诱导的特征空间的精确刻画，模型选择缺乏可解释性，而且大多是基于全数据的精确计算，缺乏可扩展性；大规模核方法大多根据先验知识或计算资源限制经验地进行模型选择，缺乏坚实理论依据。针对上述现状，提出基于积分算子谱分析的大规模核方法模型选择方法。首先，显式构造积分算子空间，从而精确刻画特征空间结构，增强模型可解释性。在此基础上，从积分算子谱稳定性和复杂度角度研究泛化误差，建立大规模积分算子谱模型选择泛化理论。最后，设计兼具可解释性、泛化理论保障和计算高效的谱模型选择准则和算法。本项目提出的积分算子谱分析方法为大规模核方法的模型选择提供一种有效途径，发展大规模核方法的理论和方法。