New Frontiers of Robust Statistics in the Era of Big Data

大数据时代稳健统计的新领域

• 批准号: 2113568
• 负责人: Zhao Ren
• 金额: $ 23.59万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113568&HistoricalAwards=false
• 关键词: New; Frontiers; Robust; Statistics; Era

Modern technologies have facilitated the collection of an unprecedented amount of features with complex structures. Although extensive progress has been made towards extracting useful information from massive data, the statistical analysis typically assumed that data are drawn without any contamination. However, in reality the data sets arising in applications such as genomics and medical imaging are usually more inhomogeneous due to either data collection process or the intrinsic nature of the data in the era of big data. For instance, in gene expression data analysis, outliers frequently arise in microarray experiments due to the array chip artifacts such as uneven spray of reagents within arrays. Compared to the recent advances in the era of big data, research in modeling and theoretical foundations for robust procedures under contamination models has fallen behind. To bridge this gap, this project seeks to develop new robust estimation and inference procedures which are rate-optimal for various contamination models as building blocks to address the modeling, theory and computational challenges. Upon completion, this work will lead to a comprehensive understanding of contamination models and have an immediate impact on various disciplines such as biology, genomics, astronomy and finance. The project also provides training opportunities for undergraduate and graduate students, and is used to enrich courses and outreach educational materials in statistics and data science.This project aims to address some of the most pressing challenges that are faced by robust procedures in high-dimensional and nonparametric contamination models. Specifically, (I) the research begins with statistical inference of low-dimensional parameters in both increasing-dimensional and high-dimensional regressions under contamination models. The PI will study the influence of contamination proportion in obtaining the root-n consistency results. Robust large-scale simultaneous inference under contamination models are also considered. (II) Next, the PI will revisit some classical nonparametric density estimation problems both under arbitrary and structured contamination distributions. The PI plans to propose rate-optimal procedures and carefully study the effect of contamination on estimation through various model indices, including contamination proportion, the structure of contamination and the choice of loss function. (III) The PI will develop a U-type robust covariance estimator under structured contamination models and provide rigorous theoretical guarantees on its rate optimality. This general robust estimator can serve as building blocks for establishing many rate-optimal procedures for structured large covariance/precision matrix estimation problems. User-friendly R packages will be developed to implement the proposed methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代技术已促进了具有复杂结构的前所未有的特征的收集。尽管从大量数据中提取有用信息方面取得了广泛的进展，但统计分析通常假定数据是没有任何污染的。但是，实际上，由于数据收集过程或大数据时代的数据的固有性质，基因组学和医学成像等应用中产生的数据集通常更加不均匀。例如，在基因表达数据分析中，由于阵列芯片伪影（例如阵列中的不均匀喷雾剂喷雾剂），在微阵列实验中经常出现异常值。与大数据时代的最新进展相比，在污染模型下为稳健程序建模和理论基础的研究落后了。为了弥合这一差距，该项目旨在制定新的强大估计和推理程序，这些估计和推理程序对于各种污染模型来说是最佳的，作为解决建模，理论和计算挑战的基础。 完成后，这项工作将导致对污染模型的全面理解，并对生物学，基因组学，天文学和金融等各种学科产生直接影响。该项目还为本科生和研究生提供了培训机会，并用于丰富统计和数据科学领域的课程和外展教育材料。该项目旨在应对高维和非参数污染模型中强大程序面临的一些最紧迫的挑战。具体而言，（i）研究始于在污染模型下增加维度和高维回归中低维参数的统计推断。 PI将研究污染比例在获得根N一致性结果中的影响。还考虑了在污染模型下同时进行强大的大规模同时推断。 （ii）接下来，PI将在任意和结构化污染分布下重新审视一些经典的非参数密度估计问题。 PI计划提出最佳速率程序，并通过各种模型指标仔细研究污染对估计的影响，包括污染比例，污染结构和损失功能的选择。 （iii）PI将在结构化污染模型下开发U型强大的协方差估计器，并根据其速率优化性提供严格的理论保证。该一般强大的估计器可以用作建立许多结构化的大协方差/精度矩阵估计问题的基本块。将开发用户友好的R软件包来实施拟议的方法。该奖项反映了NSF的法定任务，并认为使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估。

项目成果

Adaptive minimax density estimation on ℝ d for Huber’s contamination model

Huber 污染模型对 d 的自适应极小极大密度估计

• DOI: 10.1093/imaiai/iaad045
• 发表时间: 2023
• 期刊: Information and Inference: A Journal of the IMA
• 作者: Zhang, Peiliang;Ren, Zhao
• 通讯作者: Ren, Zhao

High-dimension to high-dimension screening for detecting genome-wide epigenetic and noncoding RNA regulators of gene expression

高维到高维筛选，用于检测基因表达的全基因组表观遗传和非编码 RNA 调节因子

• DOI: 10.1093/bioinformatics/btac518
• 发表时间: 2022
• 期刊: Bioinformatics
• 影响因子: 5.8
• 作者: Ke, Hongjie;Ren, Zhao;Qi, Jianfei;Chen, Shuo;Tseng, George C.;Ye, Zhenyao;Ma, Tianzhou;Alkan, ed., Can
• 通讯作者: Alkan, ed., Can