Leveraging Structural Information in Regression Tree Ensembles

利用回归树集成中的结构信息

基本信息

批准号：
1712870
负责人：
Antonio Linero
金额：
$ 10万
依托单位：
Florida State University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-09-01 至 2020-02-29
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1712870&HistoricalAwards=false
关键词：
Leveraging Structural Information Regression Tree

项目摘要

A common task in statistics is prediction; for example, a practitioner may be interested in predicting the presence of a disease given genetic information about an individual. Due to recent advances in data collection, frequently one has access to datasets which contain a massive number of predictors, but with correspondingly few subjects. This setting is generally referred to as the "big P, small n" scenario. Drawing meaningful conclusions under such circumstances is generally impossible unless the underlying data satisfy certain structural assumptions. The simplest such structural assumption is that only a small number of the predictors are relevant; in this setting, finding the useful predictors corresponds to finding a so-called "needle in a haystack." The goal of this project is to construct procedures which adapt to this, and other, structural assumptions. The project will focus on methods based on decision trees, which are flowchart-like structures in which predictions are based on whether the predictors satisfy various rules. Usually an ensemble of decision trees are constructed, with the predictions for each individual tree averaged. While decision tree ensembles are frequently used with high dimensional data, it is unclear to what extent they adapt to the structural properties of the data. This project will show that, in practice, off-the-shelf decision tree ensembling methods do not adapt to common structural assumptions, and will develop new methods which do. In addition to developing methods with strong theoretical support, this project will support the development of an R package to give practitioners easy access to our methodology. The PI will develop Bayesian methods for incorporating structural information into tree-based ensemble methods, and establish theoretically the benefit of making use of this additional information. This forms a nonparametric counterpart to the parametric approaches used in linear models, such as the lasso, graphical lasso, or group lasso; Bayesian approaches in the parametric setting include the use of variable selection priors, such as spike-and-slab priors and global-local shrinkage priors. Structural information will be incorporated by modifying the commonly used priors on decision tree ensembles so that the prior is concentrated on models which satisfy the desired structure. The PI will first investigate the theoretical properties of a sparsity inducing prior which is designed to eliminate unnecessary predictors. Sparsity here is obtained by applying a sparsity inducing Dirichlet prior to the a priori probability that a given branch is associated to a given predictor. This prior will be extended to allow for grouped variable selection in a similar manner to the group lassoby considering the class of Dirichlet tree priors, and further to accommodate graphical structures in the predictors through sparsity inducing logistic normal priors. Additionally, the PI will develop computationally efficient Markov chain Monte Carlo algorithms to fit the resulting models. Compared to existing methods, these structural priors will be shown to lead to substantial gains in predictive accuracy, and to more accurate scientific discovery.

统计中的一个常见任务是预测。例如，从业者可能有兴趣预测有关个人的遗传信息的存在。由于数据收集的最新进展，通常可以访问包含大量预测变量的数据集，但受试者很少。该设置通常称为“ Big P，Small n”场景。在这种情况下，在这种情况下得出有意义的结论通常是不可能的，除非基本数据满足某些结构性假设。最简单的结构假设是，只有少数预测因子是相关的。在这种情况下，找到有用的预测因子对应于找到所谓的“在干草堆中”。该项目的目的是构建适应此项目以及其他结构假设的程序。该项目将集中在基于决策树的方法上，这些方法是基于流程图的结构，其中预测基于预测因素是否满足各种规则。通常，构建了决策树的合奏，对每棵树的平均预测进行了预测。虽然决策树集合经常与高维数据一起使用，但尚不清楚它们在多大程度上适应了数据的结构属性。该项目将表明，在实践中，现成的决策树结合方法不能适应常见的结构假设，并且会开发出可行的新方法。除了开发具有强大理论支持的方法外，该项目还将支持开发R软件包，以使从业者易于访问我们的方法。 PI将开发贝叶斯方法，将结构信息纳入基于树的集合方法，并从理论上确定使用此其他信息的好处。这形成了与线性模型中使用的参数方法的非参数对应物，例如拉索，图形拉索或组套索；参数设置中的贝叶斯方法包括使用可变选择先验，例如尖峰和slab先验和全球本地收缩先验。结构信息将通过在决策树组合上修改常用的先验来纳入结构信息，以便先验集中于满足所需结构的模型。 PI将首先研究引起稀疏性的理论特性，以消除不必要的预测因子。稀疏性是通过在先验概率之前施加诱导dirichlet的稀疏性来获得的，即给定分支与给定的预测变量相关联。该先验将扩展，以允许以类似于Dirichlet Tree先验的类别的方式进行分组的变量选择，并通过诱导Logistic正常先验的稀疏性来适应预测指标中的图形结构。此外，PI将开发计算高效的马尔可夫链蒙特卡洛算法以适合所得模型。与现有方法相比，这些结构先验将显示出可观的预测准确性和更准确的科学发现。