CIF: RI: Small: Information-theoretic measures of dependencies and novel sample-based estimators

CIF：RI：小：依赖性的信息论测量和新颖的基于样本的估计器

• 批准号: 1815535
• 负责人: Sewoong Oh
• 金额: $ 45万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2019-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815535&HistoricalAwards=false
• 关键词: CIF; RI; Small; Information; theoretic; CIF; RI; Small; Information; theoretic

Measures of dependencies play central roles in discovering associations between variables that leads to scientific discoveries. In practice, analysts need to compute these measures from data, which can be challenging. The standard estimators can fail when, for example, the data has a mixture of continuous and discrete variables, or when the data lies on a complex space with abundant boundaries. The aim of this project is to address practical issues in estimating measures of dependencies, and provide novel estimators to overcome these challenges. The success of the proposed work will result in novel estimators for discovering new aspects of data. The immediate impact is in two specific contexts: discovering correlations in biological datasets and analyzing the inner-workings of deep neural networks; the lasting impact will be in diverse fields including genomic, biology, machine learning, and artificial intelligence. This project also integrates research with education through the creation of a graduate course on statistical learning. In addition, the project will offer undergraduates the opportunity to be involved in research.This proposal addresses two fundamental questions: designing novel estimators for information theoretic measures and designing novel estimators for modern measures of correlation that is defined as a solution of optimization problems. In the former, two major challenges are addressed: variables of mixed type (continuous and discrete) and boundary biases. Borrowing techniques from local log-likelihood density estimators, nearest neighbor methods, and order statistics, this leads to a new estimator that can adapt to the local geometry of the distributions in a principled way, that improves significantly over existing estimators. In modern data analysis, several measures of correlations are naturally defined as solutions of optimization problems, making them challenging to estimate. This proposal aims to provide a principled approach and propose a new estimator borrowing insights from importance sampling and nearest neighbor methods. The proposed framework is applied to estimate hypercontractivity ratio, an information theoretic quantity that captures hidden correlations in the data and is naturally defined as a solution of an infinite dimensional optimization. The proposed measure of hypercontractivity is shown to discover potential correlations that other standard measures are not able to, in canonical synthetic examples and real datasets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

依赖关系的度量在发现导致科学发现的变量之间的关联中起着核心作用。实际上，分析师需要从数据中计算这些措施，这可能具有挑战性。例如，当数据具有连续和离散变量的混合物，或者数据位于具有丰富边界的复杂空间时，标准估计器可能会失败。该项目的目的是解决估计依赖措施的实用问题，并提供新颖的估计器来克服这些挑战。拟议工作的成功将导致新的估计量，以发现数据的新方面。直接影响是在两个特定的情况下：发现生物数据集中的相关性并分析深神经网络的内部工作；持久的影响将在包括基因组，生物学，机器学习和人工智能在内的各个领域。该项目还通过创建有关统计学习的研究生课程将研究与教育融为一体。此外，该项目将为本科生提供参与研究的机会。该提案解决了两个基本问题：针对信息理论措施设计新颖的估计器，并为现代相关性衡量标准设计新颖的估计量定义为优化问题解决方案。在前者中，解决了两个主要挑战：混合类型（连续和离散）和边界偏见的变量。借入局部对数可能性密度估计器，最近的邻居方法和顺序统计的借用技术，这导致了一个新的估计器，该估计器可以以原则上的方式适应分布的局部几何形状，从而高于现有估计器。在现代数据分析中，几种相关性的度量自然定义为优化问题的解决方案，使它们具有挑战性。该建议旨在提供一种原则性的方法，并提出一种新的估计量借用的见解，从重要性抽样和最近的邻居方法中提出。所提出的框架应用于估计超收缩比，该信息理论量捕获数据中隐藏的相关性，并且自然定义为无限尺寸优化的解决方案。提出的超扣除率衡量标准显示出在规范合成示例和实际数据集中其他标准措施无法采取的潜在相关性。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力优点和更广泛的影响审查标准通过评估来进行评估的。