Collaborative Research: Learning and forecasting high-dimensional extremes: sparsity, causality, privacy

协作研究：学习和预测高维极端情况：稀疏性、因果关系、隐私

• 批准号: 2310974
• 负责人: Gennady Samorodnitsky
• 金额: $ 20万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-15 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310974&HistoricalAwards=false
• 关键词: Collaborative; Research; Learning; forecasting; dimensional

The principal goal of this research project is to learn how to forecast future extreme observations and to assess their impact. On an almost daily occurrence, the public is inundated with news accounts related to extreme observations arising from extraordinary climatic events from extended and severe droughts to extraordinary precipitation records, to record heat waves that have reached virtually every region of the US in one form or another. These extreme events appear unexpectedly, can be dangerous and occur in combinations that may or may not be coincidental. Do tropical storms become more deadly as global temperatures rise? Does extreme violence become more widespread as the economic conditions worsen? Questions of this type are studied by climate scientists and social scientists respectively, but statistical and probabilistic analysis of extreme values is an indispensable ingredient in any analysis. Modern statistical analysis of extremes is both blessed by the deluge of the amount of available data and cursed by this deluge. The available data are often high dimensional and contaminated. The necessity of quick forecast of future extremes and corresponding policy updates require online analysis of extremes. This research aims to evaluate causal impacts of various factors from a potentially large array of variables including changing environmental conditions, demographic movements within the US, changing landscapes, and changing economic conditions, on the frequency and magnitude of extreme events. From many variables, the hope is to produce methodology to extract the important features in the data that have a direct impact on describing and predicting extremes. This research also revolves around the notion of differential privacy and aims to develop tools for releasing global characteristics of a data set without revealing individual level information. The focus of this research will be related to developing differential privacy procedures that are tailored to extreme value characteristics of large data sets, which is challenging because extreme observations are precisely the ones that reveal the most individual information. An overarching objective of this research project is to adapt modern statistical learning tools to the problem of forecasting extremes. Learning the structure of extremes presents difficult challenges due to both a limited number of extreme data and to the scarcity of extremal labels. One approach is to develop methods for detecting nonlinear sets of much smaller dimension that can provide an adequate description of extremes in high dimensions. A main thrust of this research is to develop powerful modern learning techniques (such as graph-based learning methods and kernel principal component analysis) that allow one to determine the extremal support from the data. A second main thrust of this research centers on the issue of causality in both small and large dimensional problems. In the most basic form, a set of variables X is said to be tail causal to a dependent vector Y if certain changes in X (sometimes themselves extreme but not always so) impact the tail behavior of Y. The potential outcomes framework for causality of extreme events will be a major focus in this proposal’s research agenda. A third main thrust of this research is about differential privacy in the context of extremes, which provides tools for releasing global characteristics of a data set without revealing individual level information. This is achieved by modifying the data before releasing it and, in particular, randomizing it, in such a way that the output of the procedure does not depend too much on any specific observation while still allowing for statistical inference for certain characteristics of the original data set.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究项目的主要目标是了解如何预测未来的极端观测并评估其影响。公众几乎每天都会被与从长期严重干旱到极端气候事件有关的极端观测相关的新闻报道所淹没。非凡的降水记录，记录了几乎以某种形式到达美国每个地区的热浪。这些极端事件的出现可能是危险的，并且可能是巧合的，也可能不是巧合的，热带风暴会变得更加致命。随着全球气温上升？随着经济状况恶化，暴力变得更加普遍？气候科学家和社会科学家分别研究了此类问题，但极端值的统计和概率分析是任何分析中不可或缺的组成部分。现有数据往往是高维度的并且受到污染。快速预测未来极端情况和相应政策更新的必要性需要对极端情况进行在线分析。各种因素的影响来自于潜在的大量变量，包括不断变化的环境条件、美国境内的人口流动、不断变化的景观和不断变化的经济条件，与极端事件的频率和强度有关，希望能够从许多变量中提取出数据中的重要特征。这项研究还围绕差异隐私的概念，旨在开发在不泄露个人级别信息的情况下发布数据集全局特征的工具。针对大数据极值特征量身定制的差分隐私程序集，这是具有挑战性的，因为极端观察恰恰揭示了最个体的信息，该研究项目的首要目标是使现代统计学习工具适应预测极端的问题，因为由于以下原因，学习极端的结构面临着困难的挑战。一种方法是开发用于检测更小维度的非线性集的方法，该方法可以提供对高维度极值的充分描述。强大的现代学习技术（例如基于图的学​​习方法和核主成分分析）使人们能够确定数据的极值支持这项研究的第二个主要重点是小维度和大维度问题中最基本形式的因果关系问题。如果 X 的某些变化（有时本身是极端的，但并非总是如此）影响 Y 的尾部行为，则一组变量 X 被认为是依赖向量 Y 的尾部因果关系。极端事件因果关系的潜在结果框架将是一个主要因素重点关注本提案的研究议程。这项研究的主旨是关于极端情况下的差异隐私，它提供了在不泄露个人级别信息的情况下发布数据集全局特征的工具，这是通过在发布数据之前对其进行修改，特别是对其进行随机化来实现的。 ，以这样的方式，程序的输出不会过多地依赖于任何特定的观察，同时仍然允许对原始数据集的某些特征进行统计推断。该奖项反映了 NSF 的法定使命，并通过评估被认为值得支持利用基金会的智力优势和更广泛的影响审查标准。