Foundations of High-Dimensional and Nonparametric Hypothesis Testing

高维和非参数假设检验的基础

• 批准号: 2113684
• 负责人: Sivaraman Balakrishnan
• 金额: $ 25万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113684&HistoricalAwards=false
• 关键词: Foundations; Dimensional; Nonparametric; Hypothesis; Testing

Statistical inferential tools are the main export from the discipline of statistics to the empirical sciences, serving as the primary lens through which natural scientists interpret observations and quantify the uncertainty of their conclusions. However, in the analysis of modern large datasets the most common inferential tools available to us are fraught with pitfalls, often requiring various technical conditions to be checked before their valid application. This in turn has led to misuse of the inferential tools and subsequent misinterpretation of results. This research project will aim to address this issue by developing and analyzing new user-friendly methodologies for statistical inference in complex settings. The methods we develop will be broadly applicable to a wide variety of challenging inferential problems in the physical and biological sciences, will eliminate the need to verify technical conditions, and will ultimately be robust in their application. The principal and co-principal investigators will be involved in advising and mentoring graduate students, in curricular and course development, and in integrating the project with a research group on Statistical Methods in the Physical Sciences (STAMPS).This project will advance our understanding of high-dimensional and non-parametric inference along three frontiers. Firstly, we aim to develop statistical inferential tools for irregular models, which are valid under weak conditions. Our particular focus will be on mixture models, and on methods which use sample-splitting to avoid strong regularity conditions. Secondly, we will show that our methods achieve these strong guarantees at a surprisingly small statistical price. To rigorously quantify the statistical price paid for avoiding strong regularity conditions we will use minimax theory. However, standard minimax theory, in many cases, does not adequately capture the difficulty of statistical inference since the difficulty of inference can vary significantly across the parameter space. A more refined theory -- called local minimax theory -- leads to a more accurate picture, and we will study our methods via this lens. Finally, we will address the problem of conditional independence (CI) testing. Despite its central role in regression diagnostics, and in the study of probabilistic graphical models, the task of CI testing and its intrinsic difficulty is poorly understood. We will address two fundamental aspects of CI testing, by studying methods to appropriately calibrate CI tests, and by developing and analyzing powerful new CI tests.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

统计推论工具是从统计学学科到经验科学的主要出口，它是自然科学家解释观察结果并量化其结论不确定性的主要镜头。但是，在对现代大型数据集的分析中，我们可用的最常见推论工具充满了陷阱，通常需要在其有效应用之前检查各种技术条件。反过来，这导致滥用了推论工具和随后对结果的误解。该研究项目的目的是通过开发和分析复杂设置中的统计推断的新的用户友好方法来解决这一问题。我们开发的方法将广泛适用于物理和生物科学中各种具有挑战性的推论问题，将消除验证技术条件的需求，并最终将在其应用方面变得强大。校长和联合首席研究人员将参与建议和指导研究生，课程和课程发展，并将该项目与物理科学统计方法的研究小组（邮票）整合在一起。该项目将提高我们对沿三个边境沿三个边境的高维和非参数定义的理解。首先，我们旨在为不规则模型开发统计推断工具，这些工具在弱条件下是有效的。我们的特殊重点将放在混合模型上，以及使用样品分解以避免强大规律性条件的方法。其次，我们将证明我们的方法以惊人的统计价格实现了这些强大的保证。为了严格量化用于避免强大规律性条件的统计价格，我们将使用Minimax理论。但是，在许多情况下，标准最小值理论并不能充分捕获统计推断的难度，因为推理的难度在整个参数空间中可能会有很大差异。一种更精致的理论（称为局部的最小值理论）导致了更准确的图片，我们将通过此镜头研究我们的方法。最后，我们将解决条件独立性（CI）测试的问题。尽管它在回归诊断中的核心作用以及概率图形模型的研究，但CI测试及其内在难度的任务知之甚少。我们将通过研究适当校准CI测试的方法以及开发和分析强大的新CI测试的方法来解决CI测试的两个基本方面。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的审查标准来通过评估来通过评估来支持的。

项目成果

Domain Adaptation under Open Set Label Shift

• DOI: 10.48550/arxiv.2207.13048
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: S. Garg;Sivaraman Balakrishnan;Zachary Chase Lipton

Semiparametric Counterfactual Density Estimation

• DOI: 10.1093/biomet/asad017
• 发表时间: 2021-02
• 期刊: Biometrika
• 影响因子: 2.7
• 作者: Edward H. Kennedy;Sivaraman Balakrishnan;L. Wasserman

RLSbench: Domain Adaptation Under Relaxed Label Shift

• DOI: 10.48550/arxiv.2302.03020
• 发表时间: 2023-02
• 作者: S. Garg;Nick Erickson;J. Sharpnack;Alexander J. Smola;Sivaraman Balakrishnan;Zachary Chase Lipton

Domain Adaptation under Missingness Shift – Tackling Underreporting

缺失转变下的领域适应 — 解决漏报问题

• 发表时间: 2023
• 期刊: AISTATS
• 作者: Zhou, Helen;Balakrishnan, Sivaraman;Lipton Zachary
• 通讯作者: Lipton Zachary

Statistical Guarantees for Local Spectral Clustering on Random Neighborhood Graphs

随机邻域图上局部谱聚类的统计保证

• DOI: 10.48550/arxiv.1911.09714
• 发表时间: 2021
• 期刊: Journal of machine learning research
• 影响因子: 6
• 作者: Green, Alden;Balakrishnan, Sivaraman;Tibshirani, Ryan
• 通讯作者: Tibshirani, Ryan