Collaborative Research: Learning and forecasting high-dimensional extremes: sparsity, causality, privacy

协作研究：学习和预测高维极端情况：稀疏性、因果关系、隐私

• 批准号: 2310973
• 负责人: Richard Davis
• 金额: $ 25万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-15 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310973&HistoricalAwards=false
• 关键词: Collaborative; Research; Learning; forecasting; dimensional

The principal goal of this research project is to learn how to forecast future extreme observations and to assess their impact. On an almost daily occurrence, the public is inundated with news accounts related to extreme observations arising from extraordinary climatic events from extended and severe droughts to extraordinary precipitation records, to record heat waves that have reached virtually every region of the US in one form or another. These extreme events appear unexpectedly, can be dangerous and occur in combinations that may or may not be coincidental. Do tropical storms become more deadly as global temperatures rise? Does extreme violence become more widespread as the economic conditions worsen? Questions of this type are studied by climate scientists and social scientists respectively, but statistical and probabilistic analysis of extreme values is an indispensable ingredient in any analysis. Modern statistical analysis of extremes is both blessed by the deluge of the amount of available data and cursed by this deluge. The available data are often high dimensional and contaminated. The necessity of quick forecast of future extremes and corresponding policy updates require online analysis of extremes. This research aims to evaluate causal impacts of various factors from a potentially large array of variables including changing environmental conditions, demographic movements within the US, changing landscapes, and changing economic conditions, on the frequency and magnitude of extreme events. From many variables, the hope is to produce methodology to extract the important features in the data that have a direct impact on describing and predicting extremes. This research also revolves around the notion of differential privacy and aims to develop tools for releasing global characteristics of a data set without revealing individual level information. The focus of this research will be related to developing differential privacy procedures that are tailored to extreme value characteristics of large data sets, which is challenging because extreme observations are precisely the ones that reveal the most individual information. An overarching objective of this research project is to adapt modern statistical learning tools to the problem of forecasting extremes. Learning the structure of extremes presents difficult challenges due to both a limited number of extreme data and to the scarcity of extremal labels. One approach is to develop methods for detecting nonlinear sets of much smaller dimension that can provide an adequate description of extremes in high dimensions. A main thrust of this research is to develop powerful modern learning techniques (such as graph-based learning methods and kernel principal component analysis) that allow one to determine the extremal support from the data. A second main thrust of this research centers on the issue of causality in both small and large dimensional problems. In the most basic form, a set of variables X is said to be tail causal to a dependent vector Y if certain changes in X (sometimes themselves extreme but not always so) impact the tail behavior of Y. The potential outcomes framework for causality of extreme events will be a major focus in this proposal’s research agenda. A third main thrust of this research is about differential privacy in the context of extremes, which provides tools for releasing global characteristics of a data set without revealing individual level information. This is achieved by modifying the data before releasing it and, in particular, randomizing it, in such a way that the output of the procedure does not depend too much on any specific observation while still allowing for statistical inference for certain characteristics of the original data set.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目的主要目标是学习如何预测未来的极端观察并评估其影响。几乎每天发生，公众被新闻报道淹没了与从延伸和严重干旱到非凡降水记录的非凡气候事件引起的极端观察有关的新闻报道，以记录以一种或另一种形式到达美国每个地区的热浪。这些极端事件显然是出乎意料的，可能是危险的，并且发生在可能是或可能不是偶然的组合中。随着全球温度升高，热带风暴会变得更加致命吗？随着经济状况恶化，极端暴力会变得更加普遍吗？这种类型的问题分别由气候科学家和社会科学家研究，但是在任何分析中，对极值的统计和概率分析都是必不可少的成分。对极端的现代统计分析都受到了可用数据数量的洪水的祝福，又受到了这种洪水的诅咒。可用的数据通常是高维和污染的。快速预测未来极端和相应政策更新的必要条件需要对极端的在线分析。这项研究旨在评估各种因素的因果影响，这些因素从可能的大量变量，包括不断变化的环境条件，美国境内的人口运动，不断变化的景观以及不断变化的经济状况，对极端事件的频率和幅度的变化。从许多变量中，希望产生方法来提取数据直接影响和预测极端的重要特征。这项研究还围绕着差异隐私的概念，旨在开发用于释放数据集的全球特征的工具，而不揭示个人级别的信息。这项研究的重点将与开发针对大型数据集的极端价值特征量身定制的差异隐私程序有关，这是挑战的，因为极端观察恰恰是揭示最个人信息的差异性数据集。该研究项目的总体目标是将现代统计学习工具调整为预测极端问题。学习极端的结构带来了艰难的挑战，这既是极限数据和极端标签的稀缺性。一种方法是开发用于检测较小尺寸的非线性集的方法，这些尺寸可以为高维度提供足够的极端描述。这项研究的主要目的是开发强大的现代学习技术（例如基于图形的学习方法和内核主成分分析），以确定数据中的极端支持。这项研究的第二个主要力量集中在大小问题中的因果关系问题上。在最基本的形式中，如果X的某些变化（有时是极端但并非总是如此），则据说一组变量X是依赖向量Y的尾巴，影响了Y。极端事件的因果关系的潜在结果框架将是该提案研究议程的主要重点。这项研究的第三个主要目的是关于极端情况下的差异隐私，该隐私提供了用于释放数据集的全球特征的工具，而无需揭示个人级别的信息。这是通过在发布数据之前修改数据，尤其是将数据随机化的方法来实现的，以使得该过程的输出不大取决于任何特定的观察，同时仍允许对原始数据集的某些特征进行统计推断。该奖项反映了NSF的法定任务，反映了通过使用基础的智力效果和广泛的评估来评估的支持，并将其视为珍贵的支持。