A Novel Statistical Framework for Big Data Prediction

用于大数据预测的新型统计框架

• 批准号: 1513408
• 负责人: Shaw-Hwa Lo
• 金额: $ 30万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1513408&HistoricalAwards=false
• 关键词: Novel; Statistical; Framework; Big; Data; Novel; Statistical; Framework; Big; Data

Recent advances in genome-wide association studies (GWAS) have led to both an increase in the size of genetic data available and identification of important genetic variants responsible for a variety of diseases. Prediction for these genetic diseases has also become of paramount importance. However, prediction for big data such as GWAS is not trivial. A key obstacle in big data prediction is identifying (perhaps a small number of) variable sets that lead to good prediction when variable dimensionality can be extremely large. The project explores why a common approach towards prediction can often fail to deliver strong prediction rates. A novel, interaction-based and prediction-oriented approach to extracting hidden information contained in big data will be investigated. To improve prediction, a new criterion to guide the selection of variable sets will be developed.Prioritizing predictivity, not significance, requires using the correct estimates of prediction rates and developing predictivity-based criteria to evaluate variable sets. The project offers a novel theoretical framework by characterizing what makes for highly predictive variable sets, and providing fundamental work towards a new criterion to identify these sets. In the framework of this research project, variable sets have theoretical (true) levels of predictivity, which can be estimated with appropriately designed sample-based measures. This framework is the first that seeks to develop estimators specific to a criterion of predictivity. Additionally, methods that encompass both marginal and joint effects will be investigated, and a candidate measure of predictivity will be studied. Four real data examples are analyzed to illustrate how final predictors found via the new approach compare to other approaches in the current literature.

全基因组关联研究（GWAS）的最新进展既导致可用遗传数据的大小也增加了，并且鉴定了负责各种疾病的重要遗传变异。 这些遗传疾病的预测也变得至关重要。 但是，大数据（例如GWAS）的预测并不微不足道。 大数据预测中的一个关键障碍是识别（也许是少数）变量集，这些变量集会导致良好的预测，而当变化尺寸可能非常大时。 该项目探讨了为什么通用预测方法通常无法提供强大的预测率。 将研究一种基于相互作用的新型和以预测为导向的方法，用于提取大数据中包含的隐藏信息。 为了改善预测，将开发一个新标准来指导选择变量集的选择。对预测性而不是显着性的优化，需要使用正确的预测率估算值并开发基于预测性的标准来评估变量集。 该项目通过表征使高度预测性变量集的原因，并为确定这些集合的新标准提供基本工作，从而提供了一种新颖的理论框架。 在该研究项目的框架中，可变集具有理论（真）预测性水平，可以通过适当设计的基于样本的测量来估算。 该框架是第一个试图开发特定预测性标准的估计器的框架。 另外，将研究涵盖边际和关节效应的方法，并将研究预测性的候选方法。 分析了四个实际数据示例，以说明如何通过新方法与当前文献中的其他方法相比找到最终预测因子。